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Preface 
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Others  who  offered  valuable  comments  included  David  Plocher  and  John  Mercer  of  the 
Senate  Committee  on  Governmental  Affairs,  Donald  Goldberg  and  Kevin  Sabo  of  the 
House  Committee  on  Government  Operations,  Denise  Fantone  and  John  Kamensky  of 
the  General  Accounting  Office,  Harry  Hatry  of  the  Urban  Institute,  and  Professor  Roy 
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pated  in  the  case  studies  included  in  this  report  and  who  reviewed  sections  of  the  re¬ 
port  for  thoroughness  and  accuracy. 

Sherwood  D.  Kohn  edited  the  manuscript,  with  editorial  assistance  from 
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Summary 


Recent  events  have  focused  attention  on 
the  measurement  of  performance  by 
federal  agencies,  and  the  feasibility  of 
applying  those  measures  to  the  federal  bud¬ 
get  process.  These  efforts  have  one  concept  in 
common:  the  reasonable  notion  that  federal 
agencies  should  be  able  to  develop  measures 
of  program  success,  and  that  these  measure¬ 
ments  would  be  useful  to  managers  and  other 
policymakers.  At  least  three  separate  efforts 
are  proceeding  in  that  direction:  the  applica¬ 
tion  of  federal  financial  management  re¬ 
forms;  passage  by  the  Congress  of  legislation 
covering  performance  measurement;  and  the 
inception  of  an  effort  to  review  the  operation 
of  the  executive  branch  (embodied  primarily 
in  Vice  President  Gore's  review  of  agencies' 
performance). 


Measuring  Government 
Performance  and 
Applying  the  Measures 
to  the  Budget 

Measures  of  performance  are  inherently  dif¬ 
ferent  in  the  public  and  private  sectors.  In  the 
private  sector,  the  primary  measure  of  perfor¬ 
mance  for  an  organization  as  a  whole  is  profit. 
The  greater  the  profitability  of  a  private  firm, 
the  better  that  firm  is  performing.  Public 
agencies  have  no  such  accepted  measure  of 
performance.  Performance  must  be  judged 


against  the  goals  of  a  public  program.  Goals 
differ  from  program  to  program,  and  even 
within  a  program  there  may  be  disagreement 
about  the  precise  nature  of  its  mission. 

It  is  more  difficult,  therefore,  for  public 
agencies  to  determine  with  any  certainty  what 
they  are  trying  to  accomplish.  In  addition, 
once  agencies  have  determined  what  they 
should  be  accomplishing,  it  is  perhaps  even 
more  challenging  to  measure  progress  toward 
those  goals.  In  fact,  government  agencies 
have  usually  focused  on  measures  of  input  and 
output-resources  used  and  activities  per¬ 
formed.  An  emphasis  on  performance  requires 
the  use  of  more  measures  of  outcomes-the 
broader  results  that  are  achieved  because  of 
the  agency's  activities.  The  differences  be¬ 
tween  measuring  public-  and  private-sector 
performance  are  less  clear  at  the  suborganiza- 
tional  level.  Conversely,  entities  in  both  pri¬ 
vate  and  public  sectors  share  the  difficulty  of 
measuring  the  success  of  such  activities  as 
personnel  administration  or  research  and 
development. 

Even  more  difficult  than  measuring  public- 
sector  performance  is  finding  a  way  to  apply 
these  performance  measures  to  the  allocation 
or  management  of  resources  in  the  public  sec¬ 
tor.  Budgetary  linkages  can  be  developed  in 
three  different  ways.  The  first  is  by  budgeting 
based  on  performance.  That  is,  traditional 
line-item  budgets  could  be  replaced  by  a  sys¬ 
tem  in  which  program  managers  have  consid¬ 
erable  flexibility  in  managing  their  resources 
but  are  held  accountable  for  program  results. 
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That  type  of  system  is  the  goal  of  many  people 
who  advocate  performance  measurement  as  a 
tool  that  can  transform  the  nature  of  the  fed¬ 
eral  budget  process.  Second,  even  if  decisions 
about  govemmentwide  allocation  of  resources 
are  not  affected,  agencies  may  find  measure¬ 
ment  valuable  for  improving  management  of  a 
given  level  of  resources,  regardless  of  whether 
their  use  results  in  a  significant  shift  of  re¬ 
sources  from  one  program  or  agency  to  an¬ 
other.  Third,  measurement  may  be  used  for  fi¬ 
nancial  reporting.  In  this  case,  the  govern¬ 
ment  or  its  agencies  can  use  performance  mea¬ 
sures  to  report  on  their  accomplishments. 


Local,  State,  and 
International 
Experiences  with 
Performance-Based 
Budgeting 

Much  of  the  impetus  for  performance-based 
budgeting  comes  from  those  who  believe  that 
it  has  been  successfully  applied  in  local,  state, 
and  international  governments  and  that  the 
concept  should  therefore  be  used  by  the  U.S. 
government.  The  Congressional  Budget  Of¬ 
fice  (CBO)  reviewed  these  experiences  in  gen¬ 
eral,  as  well  as  specifically  through  selected 
case  studies.  In  addition  to  using  work  done 
by  the  General  Accounting  Office  (GAO)  and 
other  researchers,  CBO  made  site  visits  to  four 
local  governments  (Dayton,  Ohio;  Charlotte, 
North  Carolina;  St.  Petersburg,  Florida;  and 
Portland,  Oregon)  and  two  state  governments 
(Florida  and  Oregon)  that  are  currently  using 
performance  measurement.  In  addition,  CBO 
examined  the  experiences  of  several  other  na¬ 
tions  that  are  measuring,  reporting,  or  bud¬ 
geting  on  the  basis  of  performance. 

Although  all  of  the  units  of  government 
studied  use  some  form  of  performance  mea¬ 
sures,  most  are  focused  on  the  activities  of 
agencies  rather  than  on  results.  In  general, 
the  conclusions  suggest  little  evidence  of  the 


much-touted  advances  in  performance-based 
budgeting  in  local,  state,  and  foreign  govern¬ 
ments.  Performance  measures  have  a  limited 
ability  to  influence  the  allocation  of  resources, 
but  they  benefit  management  and  financial 
reporting.  State  government  success  stories  in 
performance  budgeting  are  particularly  diffi¬ 
cult  to  find,  a  fact  that  may  be  related  to  the 
relative  newness  of  state  performance  mea¬ 
surement.  Neither  CBO’s  visits  to  Florida  and 
Oregon  sites  nor  in-depth  studies  by  GAO  of 
five  other  states  identified  any  effect  on  the  al¬ 
location  of  resources  by  the  government  in  any 
state  currently  using  performance  measures.1 
At  all  levels  of  government,  however,  perfor¬ 
mance  measures  are  used  in  individual  agen¬ 
cies  to  influence  the  use  of  resources  and  are  a 
valuable  management  tool.  Further,  the  Gov¬ 
ernmental  Accounting  Standards  Board  has 
influenced  some  governments  to  expand  the 
use  of  performance  measures  for  financial 
reporting. 


Past  Federal  Efforts  to 
Link  Performance 
and  Budgeting 

The  current  effort  to  improve  the  linkage  of 
performance  measurement  and  budgeting  is  a 
logical  successor  to  three  similar  attempts  by 
the  federal  government  during  the  past  40 
years,  namely  performance  budgeting,  the 
planning-programming-budgeting  system  (al¬ 
so  known  as  PPBS),  and  zero-based  budgeting 
(ZBB).  At  least  two  of  these  (PPBS  and  ZBB) 
attempted  to  budget  on  the  basis  of  program 
results.  In  general,  these  systems  have  fallen 
short  of  their  goals.  In  particular,  the  substan¬ 
tial  effort  that  went  into  these  systems  failed 
to  change  the  way  that  federal  resources  were 
allocated. 


1.  General  Accounting  Office,  Performance  Budgeting: 
State  Experiences  and  Implications  for  the  Federal  Gov¬ 
ernment.  GAO/AFMD-93-41  (February  1993V 
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The  designers  of  each  of  these  reforms  tried 
to  mandate  a  solution  to  federal  budgeting 
and  management  by  linking  the  budget  to  in¬ 
tended  outcomes.  Several  important  lessons 
emerged  from  their  failures.  First,  such  bud¬ 
geting  systems  might  prove  antithetical  to 
traditional  incremental  budgeting,  and  might 
be  resisted  by  those  who  have  some  stake  in 
the  process  that  is  already  in  place.  Second,  it 
is  important,  if  such  efforts  are  to  succeed,  to 
reach  agreement  on  goals  and  objectives  and 
to  involve  key  staff  who  will  be  putting  the 
system  into  effect.  Third,  each  of  the  systems 
requires  a  great  deal  of  data  in  order  to  sur¬ 
vive.  Much  of  the  data  collected  in  the  earlier 
efforts  were  never  used,  which  suggests  that  it 
is  necessary  to  think  through  precisely  how  in¬ 
formation  will  be  used  and  how  systems  will 
be  put  into  effect  before  setting  out  require¬ 
ments.  If  the  designers  of  future  budget  sys¬ 
tems  do  not  take  into  account  the  difficulties  of 
tying  the  measurement  of  outcomes  to  budget¬ 
ing,  no  system  is  likely  to  deliver  on  its  prom¬ 
ise. 


Current  Practices 
in  Federal  Agencies 

Before  assessing  the  possible  effect  of  chang¬ 
ing  the  practices  of  federal  agencies  and  tying 
performance  measuring  to  the  budget,  it  is  im¬ 
portant  to  understand  what  federal  agencies 
are  doing.  CBO  reviewed  the  results  of  a  GAO 
survey  of  federal  agencies  and  studied  six  fed¬ 
eral  agencies  or  programs  in  detail  to  deter¬ 
mine  whether  they  had  developed  measures  of 
performance  and  to  what  extent  the  measures 
were  tied  to  the  budget  process. 


GAO  Study  Results 

GAO  surveyed  102  federal  agencies,  each  of 
which  had  more  than  1000  employees  or  spent 
more  than  $500  million  a  year.  These  agen¬ 
cies  accounted  for  87  percent  of  all  federal  em¬ 
ployees  and  92  percent  of  total  outlays  in  fiscal 


year  1990.  Agencies  were  asked  to  report  on 
what  performance  measures  (if  any)  they  used 
as  a  part  of  their  operations  and  how  they  used 
them  in  agency  management.  The  survey  fo¬ 
cused  on  the  types  of  measures  used,  their  us¬ 
ers,  the  impetus  for  their  development,  and 
top  managers’  use  of  and  satisfaction  with 
them.2  The  responding  agencies  indicated 
that  they  used  a  wide  array  of  performance 
measures.  The  majority  of  agencies  reported 
that  their  measures  were  based  on  results. 
Agency  managers  reported  that  performance 
measures  help  with  making  budget  decisions, 
managing  programs,  assuring  accountability, 
and  measuring  program  results  or  outcomes. 

The  GAO  survey  gathered  information 
about  a  large  number  of  federal  programs,  but 
a  self-reported  survey  is  limited  in  its  ability 
to  provide  detailed  and  verifiable  information. 
In  fact,  case  studies  of  several  agencies  also  re¬ 
ported  in  the  GAO  study  uncovered  much  less 
performance  measurement  activity  than  the 
general  survey  results  indicated.  That  finding 
suggests  that  surveys  may  be  poorly  suited  to 
generating  useful  information  in  a  complex, 
hard-to-define  area  such  as  performance  mea¬ 
surement. 

CBO  Case  Studies 
of  Federal  Agencies 

CBO  also  visited  six  agencies  in  order  to  gath¬ 
er  additional  information  on  performance 
measurement.  These  agencies  were  selected 
on  the  basis  of  discussions  with  staff  oversee¬ 
ing  the  GAO  survey  and  on  reports  concerning 
which  federal  agencies  had  already  been  using 
performance  measurement.  The  six  agencies 
evaluated  were  the  Environmental  Protection 
Agency,  the  Internal  Revenue  Service,  the  De¬ 
partment  of  Labor's  Employment  and  Train¬ 
ing  Administration,  the  Farmers  Home  Ad¬ 
ministration,  the  Department  of  Defense  (in¬ 
ternal  service  activities),  and  the  Public 


2.  These  results  are  reported  in  General  Accounting  Office, 
Program  Performance  Measures:  Federal  Agency  Collec¬ 
tion  and  Use  of  Performance  Data,  GAO/GGD-92-65 
(May  4, 1992). 
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Health  Service's  Healthy  People  2000  pro¬ 
gram. 

In  general,  the  results  of  these  case  studies 
were  consistent  with  the  experiences  of  state, 
local,  and  international  governments.  CBO 
concluded,  as  did  GAO,  that  the  GAO  survey 
results  may  have  overstated  the  extent  to 
which  outcome  measures  are  being  used  in  the 
federal  government,  and  the  extent  to  which 
these  measures  are  tied  to  agency  decisions 
about  resource  allocation.  In  fact,  in  the  cur¬ 
rent  setting,  it  is  extremely  difficult  for  agen¬ 
cies  to  link  their  performance  measures  and 
the  budget  process  in  any  meaningful  way. 
None  of  the  agencies  studied  used  perfor¬ 
mance  measurement  to  make  decisions  about 
the  level  of  resources  that  the  program  ob¬ 
tained  in  the  budget  process,  although  some 
intended  to  do  so  in  the  future.  Performance 
measurement  is  used  in  some  cases  to  award 
money  to  employees  or  subunits,  but  the  basic 
aspects  of  agency  budgets  are  not  determined 
by  the  relationship  between  inputs  and 
outcomes. 

Even  in  these  six  programs,  which  were  re¬ 
garded  by  GAO  evaluators  and  other  experts 
as  some  of  the  best  examples  of  federal  perfor¬ 
mance  measurement,  gauges  of  outcome  are 
relatively  rare.  The  measure  of  job  retention 
used  by  the  Employment  and  Training  Ad¬ 
ministration  to  evaluate  the  Job  Training 
Partnership  Act  program  may  be  the  most  ef¬ 
fective,  but  even  this  measure  is  relatively 
brief  (assessing  results  13  weeks  after  the 
training  has  ended)  and  does  not  provide  data 
on  the  marginal  impact  that  this  program  has 
in  the  long  run. 

Further,  certain  governmental  tasks  or 
functions  seem  less  amenable  to  effective  mea¬ 
surement  of  performance.  Programs  that  in¬ 
volve  the  provision  of  direct  services  that  are 
under  the  agency's  control  are  easier  to  mea¬ 
sure  than  other  operations,  such  as  regulatory 
and  grant  activities.  In  general,  performance 
systems  seem  to  work  best  where  there  is  di¬ 
rect  accountability  or  clear  cause-and-effect 
relationships  between  what  the  agency  does 


(activities)  and  what  it  wants  to  happen  (out¬ 
comes). 


Prospects  for  Using 
Performance  Measures 
in  the  Budget  Process 

The  road  to  improving  federal  performance 
and  tying  its  measurement  to  the  budget  pro¬ 
cess  is  studded  with  obstacles.  Legislation 
cannot  always  bring  about  lasting  change,  but 
it  can  assist  in  some  areas,  such  as  motivating 
agencies  to  measure  results,  increasing  flexi¬ 
bility  for  program  managers,  and  mandating 
that  policymakers  receive  information  on  pro¬ 
gram  performance.  Commitment  on  the  part 
of  the  executive  branch,  however,  is  crucial  in 
making  any  real  changes  in  agency  manage¬ 
ment,  by  providing  incentives  for  agency  man¬ 
agers,  improving  information  systems,  and 
tailoring  solutions  to  individual  problems. 

Even  if  the  legislative  and  executive 
branches  were  committed  to  improving  perfor¬ 
mance  measurement  and  tying  those  mea¬ 
surements  to  the  budget  process,  two  chief  ob¬ 
stacles  would  remain.  First,  developing  mea¬ 
surements  that  accurately  reflect  the  perfor¬ 
mance  of  federal  agencies  is  difficult,  given 
that  the  success  of  so  many  programs  man¬ 
aged  by  the  federal  government  is  determined 
in  large  part  by  nonfederal  factors.  Second, 
there  is  not  enough  demand  by  policymakers 
to  change  the  way  policies  are  made  so  that 
they  are  more  responsive  to  the  measurement 
of  outcomes. 

It  is  particularly  important  to  recognize  the 
limited  payoffs  that  may  come  from  a  link  be¬ 
tween  performance  measures  and  the  budget 
process,  because  using  performance  measures 
for  allocating  government  resources  implies  a 
change  from  the  current  processes,  which  are 
heavily  focused  on  inputs,  to  a  system  that  is 
oriented  toward  outcomes.  Creating  such  a 
system  by  simply  demanding  it  is  impossible, 
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and  past  attempts  to  create  a  demand  by  flat 
have  failed.  Real  changes  in  the  budget  pro¬ 
cess  could  take  place,  however,  if  policy¬ 
makers  demanded  infoimation  on  the  rela¬ 
tionship  between  dollars  and  results.  Report¬ 
ing  on  the  results  of  government  programs  in 
government  financial  statements  can  improve 
the  capacity  to  create  such  information,  in  ad¬ 
dition  to  developing  a  demand  for  it.  But  it 
will  not  be  easy  to  create  a  demand  or  spur 
greater  use  of  performance  measures  for  allo¬ 
cating  resources.  Finally,  the  largest  poten¬ 
tial  for  real  payoffs  may  be  in  the  area  of  agen¬ 
cy  management  of  the  resources  once  they 
have  been  provided  in  the  budget  process. 

Before  taking  further  legislative  action,  the 
Congress  might  consider  exactly  how  perfor¬ 
mance  measures  could  change  the  process  of 
developing  and  applying  policies.  If  propo¬ 
nents  expect  that  mandating  performance 
measures  will  lead  to  reform  of  federal  budget¬ 
ing  and  management,  they  will  almost  cer¬ 
tainly  be  disappointed.  Commitment  on  the 
part  of  the  executive  branch  is  probably  more 
important  than  legislation,  although  legisla¬ 
tion  aimed  at  reducing  impediments  to  man¬ 
agers  might  be  helpful.  In  addition,  it  is  im¬ 
portant  to  recognize  that  several  concurrent 
efforts-both  legislative  and  executive-are  un¬ 


der  way.  The  Chief  Financial  Officers  Act,  the 
Vice  President's  National  Performance  Re¬ 
view,  the  move  to  bring  total  quality  manage¬ 
ment  to  federal  agencies,  and  the  recently 
passed  legislation  in  the  Congress  are  all  re¬ 
lated  to  performance  measurement.  Actions 
in  these  areas  should  be  coordinated  in  order 
to  avoid  duplication  or  counterproductive  ac¬ 
tivity.  In  particular,  movements  to  instill 
strategic  planning  and  the  setting  of  objec¬ 
tives  in  federal  agencies  should  predate  re¬ 
quirements  to  develop  performance  measures 
and  to  use  those  measures  for  budgeting. 

Finally,  however,  the  budget  process  is  not 
likely  to  be  changed  substantially  until  and 
unless  decisionmakers  use  information  on  pro¬ 
gram  performance  when  making  allocation 
decisions.  Having  this  information-if  good 
measurements  of  program  results  can  be 
developed-is  a  necessary,  but  not  sufficient, 
prerequisite  to  changing  policymaking.  Such 
a  change  is  likely  only  to  occur  after  the  cul¬ 
ture  of  performance  measurement  infiltrates 
the  policy  process.  For  this  reason,  the  short- 
run  emphasis  should  remain  on  developing 
performance  measures  for  agency  manage¬ 
ment  rather  than  for  use  as  a  tool  for  allocat¬ 
ing  resources. 


Chapter  One 


Performance  Measurement  and 
Budgeting:  Issues  and  Concepts 


Interest  in  measuring  government  per¬ 
formance,  and  in  using  these  measure¬ 
ments  for  budgeting,  is  not  new. 
Elected  officials,  government  agencies,  and 
other  concerned  individuals  have  long  been 
interested  in  linking  funding  to  results.  But 
measuring  results  in  the  public  sector  is  more 
difficult  than  in  the  private  sector.  The  suc¬ 
cess  of  public  agencies  cannot  easily  be  re¬ 
duced  to  one  (or  even  a  few)  measures.  Con¬ 
versely,  the  ultimate  success  of  a  private  cor¬ 
poration  is  often  measured  by  its  profits.  The 
struggle  to  improve  the  measurement  of 
public-sector  performance,  nonetheless,  has 
spurred  numerous  attempts  to  establish  man¬ 
agement  and  budgeting  systems  that  are 
aimed  at  this  goal. 

Recently,  there  has  been  a  renewed  effort  to 
legislate  the  measurement  of  performance  in 
the  federal  government  and  to  tie  the  results 
of  federal  programs  more  explicitly  to  budget¬ 
ing.  Several  factors  have  contributed  to  this 
effort,  including  the  use  of  performance  mea¬ 
sures  by  state  and  local  governments  and  the 
need  to  reconsider  priorities  in  an  era  of  scarce 
public  resources.  In  addition,  the  broad  recep¬ 
tion  of  a  recent  book  about  government  perfor¬ 
mance,  which  included  many  "success  stories" 
drawn  from  state  and  local  governments,  has 
heightened  interest  in  the  topic.1  Several  fed¬ 
eral  initiatives  relating  to  performance  mea¬ 
surement  are  under  way,  including  the  carry¬ 
ing  out  of  federal  financial  management  re¬ 
forms  (such  as  the  Chief  Financial  Officers 


1.  David  Osborne  and  Ted  Gaebler,  Reinventing  Govern¬ 
ment  (Reading,  Mass.:  Addison- Wesley,  1992). 


Act),  the  introduction  of  legislation  mandat¬ 
ing  an  expanded  use  of  performance  measures, 
and  an  evaluation  of  executive  branch  activi¬ 
ties  that  is  being  coordinated  by  Vice  Presi¬ 
dent  Albert  Gore.2 

Performance  measures  can  play  several  dif¬ 
ferent  roles  in  the  budget  process,  including 
those  of  resource  allocation,  financial  manage¬ 
ment,  and  financial  reporting.  Performance 
measures  may  have  many  nonbudgetary  uses 
as  well.  They  may,  for  example,  be  used  to 
motivate  employees  to  improve  their  perfor¬ 
mance  or  productivity,  or  for  other  general 
management  purposes.  These  uses  are  largely 
beyond  the  scope  of  this  report,  which  focuses 
almost  exclusively  on  the  implications  of  per¬ 
formance  measurement  for  the  budget  process. 


Performance 
Measurement 
in  the  Public  Sector 

Measures  of  performance  are  inherently  dif¬ 
ferent  in  the  public  and  private  sectors.  In 
private-sector  organizations,  there  is  usually 
an  agreed-upon  measure  of  performance  for 
the  organization  as  a  whole-namely  profit. 
The  greater  the  profitability  of  a  private  firm, 
the  better  it  is  performing.  There  is  no  such 


2.  The  Chief  Financial  Officers  Act  (P.L.  101-576)  was 
passed  in  the  101st  Congress.  It  established  chief  finan¬ 
cial  officers  (CFOs)  in  23  federal  agencies;  these  CFOs 
are  to  focus  on  financial  accountability. 
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agreed-upon  measure  of  performance  for  pub¬ 
lic  agencies. 

Many  government  services  are  established 
because  the  market  is  felt  to  be  giving  the 
wrong  price  signals,  either  because  of  exter¬ 
nalities  or  because  the  service  has  the  charac¬ 
teristics  of  a  public  good.3  In  the  case  of  public 
programs,  then,  the  "price”  is  not  set  by  the 
market,  and  "profit"  is  not  a  relevant  mea¬ 
sure.  Performance  must  be  judged  against  the 
purpose  for  which  a  public  program  exists. 
This  purpose  differs  from  program  to  program, 
and  there  may  even  be  disagreement  about 
the  precise  nature  of  a  program’s  mission.  It  is 
difficult,  therefore,  for  public  agencies  to  de¬ 
termine  with  any  precision  what  they  are  try¬ 
ing  to  accomplish.  In  fact,  government  agen¬ 
cies  have  usually  focused  on  measures  of  input 
and  output-on  resources  used  and  activities 
performed.  An  emphasis  on  performance  re¬ 
quires  more  measures  of  outcomes-the  broad¬ 
er  results  that  are  achieved  because  of  the  ac¬ 
tivities  of  the  agency.  The  differences  between 
measuring  public-  and  private-sector  perfor¬ 
mance  are  less  clear  at  the  suborganizational 
level.  The  difficulty  of  measuring  the  success 
of  such  activities  as  personnel  administration 
or  research  and  development,  for  example,  is 
shared  by  entities  in  both  the  private  and  pub¬ 
lic  sectors. 

On  the  whole,  public-sector  performance 
measures  are  more  subjective  than  measures 
in  the  private  sector.  This  subjectivity  has  led 
to  the  development  of  more  indirect  methods 
to  measure  the  worth  and  results  of  federal 
programs  (such  as  benefit-cost  analysis)  that 
attempt  to  find  a  measure  in  the  public  sector 
that  is  analogous  to  the  profit  measure.  In  ad¬ 
dition,  measures  often  need  to  be  developed  at 
the  suborganizational  level;  they  are  often 
quite  subjective  for  both  private  and  public  or- 


3.  Both  externalities  and  public  goods  are  essential  con¬ 
cepts  in  welfare  economics.  Externalities  are  costs  or 
benefits  that  accrue  to  society  as  a  whole  but  are  not  ac¬ 
counted  for  in  the  market  transaction  for  a  given  good. 
Public  goods  have  two  characteristics  that  imply  that 
they  cannot  be  provided  efficiently  by  private  markets; 
benefits  are  provided  to  people  regardless  of  whether 
they  pay  for  the  good  or  not,  and  it  is  impossible  to  deter¬ 
mine  the  cost  that  should  be  charged  to  any  individual. 


ganizations.  In  the  public  sector,  officials  who 
seek  to  measure  the  effectiveness  of  a  program 
must  establish  the  purposes  of  government  ac¬ 
tivity  and  then  develop  measurements  of  how 
well  that  purpose  is  being  met.  Either  is  a  dif¬ 
ficult  task. 


The  ability  to  measure  performance  is  inexo¬ 
rably  related  to  a  statement  of  what  the  agen¬ 
cy  or  program  is  trying  to  accomplish.  The 
task  of  clarifying  those  goals  is  much  more  dif¬ 
ficult  for  public-sector  agencies  than  for  pri¬ 
vate  corporations. 

Public-sector  agencies  operate  in  an  envi¬ 
ronment  in  which  they  are  usually  asked  to  re¬ 
spond  to  many  actors,  including  legislative 
bodies,  elected  executives,  and  the  general 
public.  Not  all  of  these  actors  agree  on  the  ob¬ 
jectives  of  the  agency  or  program.  For  exam¬ 
ple,  there  is  often  disagreement  about  whether 
a  program  should  be  managed  in  a  way  that 
promotes  efficiency  (minimizing  costs  per  unit 
of  output)  as  a  primary  objective,  or  whether  it 
should  operate  primarily  to  provide  equal  ac¬ 
cess  to  services  to  as  many  citizens  as  possible, 
with  cost  as  a  secondary  concern.  Both  are  le¬ 
gitimate  goals,  but  they  often  conflict. 

For  example,  a  mass  transit  agency  that 
stresses  efficiency  as  a  primary  value  would 
operate  only  those  bus  routes  on  which  fares 
pay  for  the  operating  cost  of  the  route.  If  that 
same  agency  stressed  access  to  service  as  a  pri¬ 
mary  goal,  routes  would  be  operated  to  pro¬ 
vide  bus  service  to  as  much  of  the  population 
as  possible,  regardless  of  cost.  In  order  to  as¬ 
sess  performance,  one  would  need  to  know 
which  of  the  two  objectives  is  valued  most  (ef¬ 
ficiency  or  access).  In  the  private  sector,  there 
are  also  complex  multiple  outputs  of  services. 
But  prices  that  reflect  consumer  choices  and 
costs  often  combine  to  reduce  these  to  a  single 
measurement:  profit.  This  single  measure, 
which  stockholders  or  managers  can  turn  to  in 
order  to  gauge  performance  for  the  entity  as  a 
whole,  has  no  analog  in  the  public  sector. 


Setting  Objectives 
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Establishing  Measures  of 
Performance 

Setting  objectives  is  generally  only  the  first 
step.  The  next  is  to  design  measurements  that 
provide  information  to  policymakers  and  line 
managers  that  will  allow  them  to  evaluate 
their  progress  toward  agreed-upon  objectives. 
Although  many  government  agencies  collect  a 
great  deal  of  data,  these  data  have  typically 
focused  on  the  activities  of  the  agency  rather 
than  its  results  or  broader  societal  conse¬ 
quences.  Input  and  output  measures  are  typi¬ 
cally  used  by  agencies,  even  though  outcome 
and  impact  measures  would  be  more  meaning¬ 
ful  in  evaluating  programs. 

Inputs  represent  the  resources  consumed 
by  the  operation  of  a  government  program; 
they  are  ultimately  used  to  hire  personnel, 
build  facilities,  contract  for  services,  and  so  on. 
Inputs  are  easily  measured,  usually  in  terms 
of  dollars.  They  are  necessary  for  the  achieve¬ 
ment  of  objectives,  but  the  question  of  how 
many  inputs  are  required  usually  goes  unan¬ 
swered. 

Outputs  are  the  results  of  an  agency's  im¬ 
mediate  activities.  Although  these  outputs 
may  not  be  sufficient  for  the  agency  to  achieve 
its  objectives,  they  are  often  necessary  in  order 
for  the  agency  to  do  so.  These  gauges  of  out¬ 
puts  are  sometimes  also  referred  to  as  activity, 
work  load,  or  process  measures.  Unlike  in¬ 
puts,  outputs  are  often  impossible  to  translate 
into  dollars  because  there  are  no  markets  for 
most  government  activities. 

Outcomes  represent  the  broader  goals  of 
an  agency.  Outcome  measurement  should 
concern  the  extent  to  which  the  activities  (out¬ 
puts)  of  the  agency  have  an  intended  effect. 
That  is,  they  focus  not  only  on  the  work  per¬ 
formed,  but  also  on  the  results  of  that  work. 
Outcome  measurements  may  cover  activities 
that  are  largely  under  the  control  of  program 
managers,  or  they  may  extend  to  an  even 
broader  set  of  measures  (often  called  measures 
of  impact)  representing  results  that  the  agen¬ 


cy  may  influence  but  does  not  achieve  on  its 
own. 

The  most  important  distinction  stressed  in 
this  study  is  that  between  measurements  of 
output  and  outcome.  They  are  often  confused, 
but  an  understanding  of  the  difference  is  cru¬ 
cial  to  understanding  the  challenges  to  and 
the  potential  benefits  of  measuring  govern¬ 
mental  performance.  This  distinction  is  often 
clearest  when  considering  specific  measures 
related  to  individual  government  activities 
(see  Box  1). 


Obstacles  to  Measuring 
the  Performance 
of  Federal  Agencies 

It  is  certainly  desirable  to  determine  a  pro¬ 
gram's  objectives  and  then  to  devise  measure¬ 
ments  or  other  strategies  to  determine  how 
well  they  are  being  accomplished.  But  it  is  not 
necessarily  easy.  There  are  obstacles  to  better 
measuring  performance  for  all  organizations, 
including  public-sector  agencies  in  general 
and  federal  agencies  in  particular.  These  ob¬ 
stacles  include  the  difficulty  of  reaching 
agreement  on  objectives  and  priorities,  the 
complexity  of  developing  measures  of  results 
for  government  programs,  the  nature  of  the 
activities  engaged  in  by  the  federal  govern¬ 
ment,  and  the  incentives  for  participants  in 
the  process  to  obstruct  the  reporting  of  valid 
performance  measures. 

Defining  Objectives 

Because  objectives  in  the  public  sector  are 
open  to  debate  and  interpretation,  one  of  the 
greatest  obstacles  to  public  performance  mea¬ 
surement  involves  agreement  between  and 
within  the  Congress  or  the  executive  branch, 
and  with  other  significant  actors,  on  what  the 
objectives  of  the  agency  are.  A  program  may 
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also  have  multiple  objectives,  but  there  may 
be  disagreement  about  their  relative  impor¬ 
tance.  The  space  station  is  an  example.  Some 
might  view  its  objective  as  the  ability  to  sus¬ 


tain  manned  orbits  for  an  extended  period  of 
time,  leading  to  breakthroughs  in  scientific  re¬ 
search.  Others,  however,  might  view  the  pri¬ 
mary  objective  of  the  space  station  to  be  mili- 


Box  1. 

Output  or  Outcome? 

Examples  of  Performance  Measures  for  Government  Programs 

One  of  the  most  difficult  tasks  in  measuring  govern¬ 
ment  performance  is  that  of  deciding  on  terminol¬ 
ogy.  The  concept  of  inputs  is  fairly  easy  to  under¬ 
stand,  and  these  measures  do  not  differ  very  much 
from  one  program  to  another.  In  general,  inputs  re¬ 
present  the  raw  materials,  such  as  personnel,  build¬ 
ing  materials,  and  office  equipment,  that  go  into  the 
delivery  of  government  services.  There  is  consider¬ 
ably  more  confusion  between  outputs  and  outcomes. 

In  general,  in  order  to  be  classified  as  an  outcome 
measurement,  the  assessment  must  present  infor¬ 
mation  that  enables  the  policymaker  to  determine 
how  well  a  particular  program  is  operating  in  rela¬ 
tion  to  its  goals.  Measures  of  output,  however,  focus 
on  the  amount  of  work  accomplished  or  on  the  qual¬ 
ity  of  the  processes  used  to  accomplish  that  work. 

Outputs  are  sometimes  treated  as  ends  in  them¬ 
selves.  But  the  question  of  how  much  work  is  done  is 
very  often  distinct  from  the  question  of  whether  the 
work  is  achieving  a  particular  purpose.  For  this  rea¬ 
son,  if  the  purpose  of  measuring  performance  is  eval¬ 
uating  the  results  achieved  by  a  program,  outcome 
measures  are  superior  to  output  measures.  The  im¬ 
portance  of  the  distinction  between  output  and  out¬ 
come  can  best  be  demonstrated  by  looking  at  exam¬ 
ples  of  measures  from  several  different  government 
programs  (see  table). 

Public-sector  performance  measurement  often 
addresses  inputs  and  outputs  but  addresses  out¬ 
comes  less  frequently.  Most  of  the  current  support 
for  performance  measurement  comes  from  those  who 
want  to  move  away  from  input  and  output  measures 
toward  outcome  measures. 

Examples  of  Output  and  Outcome  Measures  for  Selected  Programs 

Output  Measure 

Outcome  Measures 

Elementary  and  Secondary  Education 

Student-days 

Students  graduated 

Dropout  rate 

Test  score  results 

Percentage  of  graduates  employed 

Hospitals 

Patient-days 

Average  length  of  stay 

Admissions 

Mortality  rates 

Patient  survey  results 

Readmission  rates 

Mass  Transit 

Vehicle  miles 

Number  of  passengers 

Population  served  (percent) 

Late  trips  (percent) 

Police 

Hours  of  patrol 

Crimes  investigated 

Number  of  arrests 

Rates  at  which  cases  are  cleared 

Response  time 

Citizen  satisfaction 

Public  Welfare  Programs 

Number  of  requests 

Amount  of  assistance 

Applications  processed  in  45  days 

Payment  error  rates 

Road  Maintenance 

Miles  resurfaced 

Lane-miles  improved  (percent) 
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tary  rather  than  scientific.  Still  others  might 
focus  on  the  creation  of  jobs  in  the  aerospace 
industry. 

Developing  consensus,  though  difficult,  is 
an  essential  step  in  developing  a  meaningful 
system  for  measuring  performance.  It  is  im¬ 
possible  to  design  systems  to  measure  the  per¬ 
formance  of  public-sector  programs  without 
clearly  understanding  what  the  program  is 
trying  to  accomplish.  If  a  program's  objectives 
cannot  be  determined,  performance  measures 
will  always  be  ambiguous,  if  not  superfluous. 
In  fact,  developing  the  consensus  necessary  to 
pass  legislation  may  impede  agreement  on 
goals  by  promoting  multiple  goals  that  are 
poorly  defined. 

Problems  in  Developing 
Measures  of  Program  Results 

Many  agencies  concentrate  on  measures  of 
work  load  or  activity,  since  (even  where  agree¬ 
ment  on  agency  goals  is  possible)  it  is  often  dif¬ 
ficult  to  develop  measures  of  the  ultimate  re¬ 
sults  of  a  program.  Outcome  measures  are 
particularly  difficult  to  address  because  it  is 
hard  to  find  acceptable  measures  of  the 
achievement  of  a  policy's  objectives  that  are 
under  the  control  of  program  managers. 

For  example,  the  ultimate  policy  objective 
for  a  job  training  program  may  be  to  reduce 
long-term  unemployment  through  providing 
people  with  job  skills.  Because  many  other 
factors  affect  unemployment,  it  is  very  diffi¬ 
cult  to  determine  the  policy  or  program's  ef¬ 
fect.  A  measure  of  some  intermediate  activity 
(that  is,  one  that  is  under  the  control  of  pro¬ 
gram  managers)  may  therefore  be  necessary. 
In  the  case  of  a  training  program,  such  mea¬ 
sures  have  usually  focused  on  how  many  pro¬ 
gram  participants  are  placed  in  jobs.  But 
since  this  measure  is  not  directly  related  to 
the  program's  effect  on  long-term  unemploy¬ 
ment,  this  kind  of  measure  will  necessarily  be 
imperfect. 

Many  effects  that  are  beyond  the  control  of 
program  designers  and  agency  managers  in¬ 


fluence  outcomes.  For  example,  citizens  some¬ 
times  assist  in  delivering  government  ser¬ 
vices.  This  is  often  referred  to  as  "coproduc¬ 
tion"  and  encompasses  such  activities  as  par¬ 
ticipating  in  parent-teacher  organizations  and 
recycling  waste  products.  To  the  extent  that 
these  factors,  which  are  external  to  public 
agencies,  can  influence  outcomes,  the  direct 
"production”  of  the  agency  will  misstate  the 
effect  of  its  policies.  Because  of  the  difficulty 
in  determining  the  cause  of  outcomes,  some 
people  advocate  that  government  should  focus 
only  on  outputs.*  Whether  that  is  practical 
may  well  depend  on  how  many  factors  are  be¬ 
yond  the  control  of  program  managers,  and 
whether  they  can  themselves  be  identified  and 
controlled  for.  Even  where  outcomes  are  not  a 
measure  of  program  success,  they  may  still  be 
important  as  indicators  of  broader  societal 
trends. 

A  troublesome  tension  may  result  between 
the  reliability  of  output  measures  and  the  ap¬ 
propriateness  of  outcome  measures.  On  the 
one  hand,  the  most  meaningful  measures  of 
success  for  a  program  are  measures  that  pro¬ 
vide  answers  to  the  question:  How  well  did 
the  program  do  in  solving  a  defined  societal 
problem  or  in  meeting  a  broad  social  or  politi¬ 
cal  goal?  On  the  other  hand,  the  question  that 
is  easiest  to  answer  reliably  is:  What  did  the 
agency  do?  These  measures  of  activity  are  es¬ 
sential  in  evaluating  a  program's  perfor¬ 
mance.  But  unless  the  precise  linkage  be¬ 
tween  agency  activities  and  program  results  is 
clear,  measuring  activities  will  never  provide 
a  full  understanding  of  the  true  results  of  a 
program. 

Even  if  reasonable  measures  of  program  re¬ 
sults  could  be  found,  how  would  one  know 
whether  the  goal  being  measured  is  an  appro¬ 
priate  one?  Consider  a  case  in  which  a  federal 
subsidy  is  provided  to  a  large  group  of  people 
rather  than  being  targeted  toward  individuals 
on  the  basis  of  need.  If  we  measured  how  effi¬ 
cient  the  delivery  of  the  subsidy  is,  and  how 


4.  See  Charles  T.  Clotfelter,  "The  Private  Life  of  Public 
Economics,"  Southern  Economic  Journal,  vol.  59,  no.  3 
(1993),  p.587. 
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satisfied  the  recipients  are,  we  might  know 
how  well  the  program  met  its  goals.  It  would 
not  tell  us  whether  an  alternate  design  of  the 
program  (one  in  which  a  targeted  group  of  de¬ 
serving  recipients  received  a  direct  cash  pay¬ 
ment,  for  example,  while  others  did  not)  might 
result  in  a  better  use  of  societal  resources. 
Without  giving  thought  to  whether  the  pur¬ 
poses  of  public  programs  are  appropriate,  per¬ 
formance  measures  might  badly  misinform 
decisionmakers.  That  may  be  a  problem  in  the 
case  of  public  programs,  since  legislative  bod¬ 
ies  (including  the  Congress)  may  avoid  clarify¬ 
ing  the  specific  purposes  of  a  government  pro¬ 
gram  in  order  to  obtain  enough  legislators' 
votes  to  gain  approval  of  it. 


Federal  Activities  Are  Varied 
and  Often  Difficult  to  Measure 

The  federal  government  engages  in  a  wide  va¬ 
riety  of  activities,  and  each  suggests  different 
challenges  for  the  measurement  of  perfor¬ 
mance.  There  are  several  classifications  of 
government  activities,  but  a  useful  one  is  in¬ 
cluded  in  the  Budget  of  the  United  States  Gov¬ 
ernment.  Fiscal  Year  1994.  This  classification 
compares  the  outlays  of  the  federal  govern¬ 
ment  in  five  categories:  direct  benefit  pay¬ 
ments  for  individuals  (46  percent  of  estimated 
outlays  for  fiscal  year  1993);  national  defense 
(20  percent);  grants  to  states  and  localities  (15 
percent);  net  interest  (14  percent);  and  other 
federal  operations,  including  deposit  insur¬ 
ance  (7  percent).  Of  these  categories,  only  net 
interest  seems  completely  inappropriate  for 
performance  measurement.  The  characteris¬ 
tics  of  programs  in  each  of  the  other  categor¬ 
ies,  however,  indicate  that  very  different 
strategies  would  be  needed  in  order  to  mea¬ 
sure  their  performance. 

Direct  payments  for  individuals  are  used 
primarily  by  three  programs-Social  Security, 
Medicare,  and  Medicaid.  These  programs  pro¬ 
vide  services  to  clients;  therefore,  the  task  of 
developing  performance  measures  would  prob¬ 
ably  focus  on  the  quality  with  which  these  ser¬ 
vices  were  provided  and  the  satisfaction  of 
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those  who  receive  them.  For  example,  mea¬ 
sures  of  performance  in  the  Social  Security 
program  would  include  the  timeliness  of  check 
issuance  and  the  error  rates  of  checks  (wheth¬ 
er  they  were  received  by  the  correct  recipi¬ 
ents).  Surveys  of  client  satisfaction  are  also 
useful  in  determining  the  quality  of  direct  ser¬ 
vices. 

National  defense  is  a  particularly  difficult 
area  for  which  to  develop  measures  of  ultimate 
performance,  perhaps  because  it  is  almost  im¬ 
possible  to  determine,  without  some  military 
conflict,  whether  the  objective  of  national  de¬ 
fense  is  being  achieved.  Measurement  is  fur¬ 
ther  complicated  by  the  concept,  as  some 
would  argue,  that  the  purpose  of  defense  is  not 
winning  wars  but  preventing  them.  For  this 
reason,  many  measures  of  performance  for  na¬ 
tional  defense  often  focus  on  the  quality  and 
readiness  of  military  forces,  assuming  that  a 
well-qualified,  appropriately  trained  military 
can  best  achieve  whatever  goals  are  para¬ 
mount  at  the  moment. 

Grants  to  states  and  localities  vary  in 
purpose  and  form:  that  is,  some  are  distributed 
by  formula  and  others  are  provided  at  the  dis¬ 
cretion  of  the  administering  agency.  The  ways 
in  which  they  are  administered,  however, 
make  it  difficult  to  measure  the  performance 
of  these  programs.  Many  federal  grant  pro¬ 
grams  are  administered  at  the  state  or  local 
level,  meaning  that  their  success  is  only  par¬ 
tially  under  federal  control.  Determining  per¬ 
formance  for  these  programs,  therefore,  in¬ 
volves  understanding  the  nature  of  their  ad¬ 
ministration  both  in  Washington  and  at  the 
recipient  level,  as  well  as  their  goals  and  pur¬ 
poses. 

Other  federal  operations  encompass  a 
wide  range  of  governmental  activities,  includ¬ 
ing  law  enforcement,  central  management 
and  administration,  and  regulatory  activities. 
Determining  performance  is  highly  dependent 
on  the  specific  nature  of  the  activity.  For  law 
enforcement,  measures  of  success  may  include 
the  percentage  of  crimes  solved.  For  an  agen¬ 
cy  such  as  the  Internal  Revenue  Service,  mea¬ 
surement  might  focus  on  the  dollars  collected 
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or  the  extent  of  taxpayer  compliance.  For  a 
regulatory  activity,  success  depends  in  large 
part  on  the  companion  regulatory  activities  of 
state  and  local  governments  and  on  the  re¬ 
sponse  of  regulated  organizations.  Thus,  it  is 
difficult  to  link  federal  action  and  ultimate  re¬ 
sults. 

The  main  point  is  that  the  ease  with  which 
performance  measures  can  be  developed  and 
the  ability  of  these  measurements  to  inform 
decisionmakers  differ  from  program  to  pro¬ 
gram.  In  general,  developing  measures  of  suc¬ 
cess  (or  at  least  satisfaction)  may  be  easier  for 
programs  in  which  there  is  direct  interaction 
between  the  federal  government  and  some  re¬ 
cipient  of  government  service  than  for  pro¬ 
grams  in  which  goals  are  less  clear  (such  as 
national  defense)  or  in  which  achievement  is 
controlled  by  many  different  factors  (such  as 
grants  to  state  and  local  governments).  As  a 
practical  matter,  the  federal  government 
should  not  be  treated  as  a  monolithic  entity, 
implying  that  similar  solutions  exist  to  all 
problems.  Instead,  any  search  for  perfor¬ 
mance  measures  should  be  activity-specific. 


Possible  Obstruction 
by  Participants 

People  who  would  typically  produce  or  use  per¬ 
formance  data  may  have  incentives  to  misre- 
port  it  or  fail  to  report  it  altogether.  If  federal 
employees  or  agency  managers  believe  that 
performance  data  might  be  used  to  punish 
them,  they  may  measure  only  those  things 
that  will  reflect  favorably  on  their  activities. 
Further,  requesters  or  recipients  of  perfor¬ 
mance  data  can  concentrate  on  measures  that 
will  support  predetermined  policy  positions. 
An  effort  to  identify  appropriate  measures  and 
monitor  the  reporting  of  performance  data 
may  therefore  be  crucial  to  the  success  of  any 
performance  measurement.  Obstruction  of 
performance  measures  is  less  likely  to  be  suc¬ 
cessful  in  cases  where  the  accuracy  and  appro¬ 
priateness  of  the  measurements  are  moni¬ 
tored. 


How  Can  Performance 
Measurement  Affect 
the  Budget  Process? 

Many  efforts  to  mandate  improvements  in  per¬ 
formance  measurement  focus  on  the  budget 
process.  Reforms  such  as  the  planning¬ 
programming-budgeting  system  in  the  1960s 
and  zero-based  budgeting  in  the  1970s 
stressed  the  ability  to  base  budgetary  choices 
more  explicitly  on  desired  results.  More  re¬ 
cently,  through  the  passage  of  the  Chief  Fi¬ 
nancial  Officers  (CFO)  Act  and  the  passage  of 
legislation  on  performance  measurement,  the 
Congress  is  once  again  addressing  the  issue  of 
using  performance  measures  in  the  budget 
process.5 

There  are  three  possible  advantages  to  link¬ 
ing  performance  measures  to  the  budget.  The 
first  is  the  potential  to  use  performance-based 
budgeting.  That  is,  traditional  line-item  bud¬ 
gets  could  be  replaced  by  a  system  granting 
program  managers  greater  flexibility  in  man¬ 
aging  their  resources,  but  holding  them  ac¬ 
countable  for  achieving  program  results.  That 
is  the  goal  of  many  who  advocate  performance 
measurement  as  a  tool  that  can  transform  the 
federal  budget  process.  Second,  even  if 
govemmentwide  decisions  on  resource  alloca¬ 
tion  are  not  affected,  agencies  may  find  mea¬ 
surements  valuable  for  improving  their  man¬ 
agement  of  a  given  level  of  resources,  regard¬ 
less  of  whether  their  use  results  in  a  signifi¬ 
cant  shift  of  resources  from  one  program  or 
agency  to  another.  Third,  the  government  or 
particular  agencies  may  use  performance 
measures  to  report  on  their  accomplishments. 


5.  S.  20  and  H.R.  826  are  the  Senate  and  House  versions  of 
the  Government  Performance  and  Results  Act.  H.R.  826 
passed  the  House  on  May  25, 1993.  S.  20  passed  the  Sen¬ 
ate  on  June  23,  1993.  On  July  15,  1993,  the  House 
agreed  to  the  Senate  version  of  S.  20,  clearing  the  bill  for 
the  President’s  signature. 


8  USING  PERFORMANCE  MEASURES  IN  THE  FEDERAL  BUDGET  PROCESS 


July  1993 


Allocation  of  Resources 

The  President  and  the  Congress  could  use  per¬ 
formance  information  to  make  decisions  on 
how  to  allocate  scarce  resources  among  com¬ 
peting  priorities.  For  them,  performance  mea¬ 
sures  are  most  useful  if  they  can  help  in  deter¬ 
mining  how  much  money  should  be  spent  on 
the  various  purposes  of  government:  health 
care  programs,  national  defense,  environmen¬ 
tal  regulation,  and  so  on.  Ideally,  these 
choices  would  be  aided  by  a  better  expectation 
of  what  a  new  dollar  for  national  defense 
would  buy  compared  with  spending  the  same 
dollar  on  some  other  activity,  such  as  research 
at  the  National  Institutes  of  Health. 

Using  outcome  measures  to  allocate  re¬ 
sources  is  a  substantial  deviation  from  the 
way  that  most  observers  describe  the  current 
budget  allocation  process.  Policymakers  cur¬ 
rently  are  focused  on  inputs  (usually  repre¬ 
sented  by  agency  line  items)  and  on  making 
marginal  changes  in  those  inputs  from  year  to 
year.  They  obviously  must  consider  benefits 
as  well,  but  these  assessments  of  benefits  are 
less  formal  and  comprehensive  than  assess¬ 
ment  of  costs.  This  input-focused  system  is 
substantially  different  from  one  that  would 
concentrate  on  trade-offs  between  results  of 
agency  programs  rather  than  on  adjusting  tra¬ 
ditional  line-item  budgets.  The  two  methods 
of  budgeting  imply  substantially  different  em¬ 
phases;  line-item  budgeting  focuses  almost  ex¬ 
clusively  on  how  much  money  is  being  spent, 
and  performance-based  budgeting  would  con¬ 
centrate  on  varying  levels  of  program  results 
that  might  accompany  varying  levels  of 
funding. 

Using  performance  measures  to  allocate  re¬ 
sources,  however,  is  difficult.  First,  not  only  is 
it  hard  to  develop  measures  based  on  results 
for  government  programs,  but  it  is  almost  im¬ 
possible  to  establish  common  denominators  of 
performance  among  the  activities  of  govern¬ 
ment.  For  example,  the  measures  of  success  in 
the  health  area  are  different  from  those  in  the 


defense  area.  Using  performance  measures 
for  resource  allocation  implies  knowledge  of 
how  to  compare  the  measures  for  those  two  ac¬ 
tivities  in  a  way  that  would  inform  trade-offs 
between  the  two.  But  trade-offs  between  gov¬ 
ernment  activities  are  almost  exclusively  a 
function  of  the  perceived  need  and  priority  for 
government  action.  Performance  measures 
can  aid  those  decisions  by  establishing  the  cor¬ 
rect  level  of  need,  but  they  cannot  tell  deci¬ 
sionmakers  how  much  to  provide  to  one  activ¬ 
ity  at  the  expense  of  another.  Politics-in  the 
best  sense  of  the  word-plays  an  important  and 
legitimate  role  in  budget  decisions,  even 
where  measures  of  outcomes  exist. 

Second,  the  relationship  between  perfor¬ 
mance  and  the  budget  is  not  straightforward. 
Poor  results  may  be  caused  by  the  difficulty  of 
the  problem  being  addressed  rather  than  by 
inadequacies  in  the  design  or  management  of 
a  program.  In  fact,  inadequate  funding  of  a 
program  may  cause  poor  performance,  which 
would  lead  to  the  conclusion  that  poorly  per¬ 
forming  programs  ought  to  get  more  re¬ 
sources,  and  no  increases  should  be  given  to 
agencies  that  meet  their  objectives. 

Even  if  performance  measures  cannot  be  ex¬ 
pected  to  dictate  the  level  of  resources  given  to 
one  agency  or  program  as  opposed  to  another, 
they  may  provide  useful  information  for 
decisionmakers.  For  example,  if  policymakers 
could  obtain  information  about  the  connection 
between  a  given  level  of  resources  and  a  given 
level  of  results  for  a  program,  they  might 
choose  to  provide  a  larger  budget  only  if  the 
incremental  improvement  in  outcomes  was 
judged  to  be  worth  the  additional  expenditure. 
In  this  way,  performance  measures  can  inform 
the  budget  process  without  dictating  budget 
outcomes.  Performance-based  budgeting, 
then,  is  about  transforming  the  debate  over  re¬ 
source  allocation  from  its  current  focus  on  in¬ 
puts  (dollars,  personnel,  and  so  forth)  to  focus 
on  results.  This  change  in  focus  does  not,  nor 
does  it  intend  to,  remove  politics  from  the 
process. 
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Agency  Management 
of  Internal  Resources 

Governmentwide  resource  allocation  is  not  the 
only  potential  function  of  performance  mea¬ 
surement.  It  is  useful  to  measure  performance 
better,  even  without  a  link  between  gauging 
performance  and  allocating  funds  in  a  budget. 
Agencies  may  find  measurement  valuable  for 
improving  the  use  of  a  given  level  of  resources, 
even  if  it  does  not  result  in  a  significant  shift 
of  resources  from  one  program  to  another  or 
from  one  agency  to  another.  For  example,  an 
agency  that  is  organized  geographically  could 
use  performance  measures  to  target  resources 
toward  those  regions  where  the  work  load  is 
greatest  or  where  the  problems  are  most 
acute.  Agency  subunits  may  improve  perfor¬ 
mance  if  workers  become  more  highly  moti¬ 
vated  when  their  efforts  are  being  scored  and 
assessed  by  management.  Moreover,  ties  may 
be  developed  between  the  measurement  of  or¬ 
ganizational  and  individual  performance, 
such  as  proponents  of  so-called  pay-for-perfor- 
mance  schemes  have  suggested. 

Financial  Reporting 

Performance  measures  can  also  be  used  to  re¬ 
port  the  results  of  government  activities  to 
elected  officials  and  decisionmakers.  At  the 
state  and  local  level,  the  Governmental  Ac¬ 
counting  Standards  Board  has  called  for  ser¬ 
vice  efforts  and  accomplishments  (SEA)  re¬ 
porting.  The  SEA  initiative  encourages  state 
and  local  governments  to  include  statements 
of  service  results  in  their  annual  financial  re¬ 
ports.  At  the  federal  level,  both  the  CFO  act 
and  the  Federal  Accounting  Standards  Advi¬ 
sory  Board  (FASAB)  are  expected  to  focus  on 
the  use  of  performance  measures  for  financial 
reporting.  The  CFO  act,  for  example,  requires 
agency  CFOs  to  develop  "systematic  measures 
of  performance"  for  programs  in  their  agen¬ 
cies.  Although  the  FASAB  is  charged  primar¬ 
ily  with  developing  common  standards  for 


financial  reporting,  the  need  for  performance 
measures  has  also  been  discussed  in  some  of 
the  board’s  deliberations. 


Conclusion 

Increasing  the  emphasis  on  outcome-oriented 
performance  measurement  in  government  is 
challenging.  Performance-based  budgeting  is 
even  more  challenging.  Given  the  obstacles 
and  problems  discussed  above,  it  may  be  sur¬ 
prising  to  discover  that  performance  measure¬ 
ment  is  being  used  to  affect  government  bud¬ 
geting.  Moreover,  the  gaps  between  current 
practice  (line-item  budgeting)  and  perfor¬ 
mance-based  budgeting  are  substantial.  It  is 
more  likely,  in  the  short  run,  that  perfor¬ 
mance  measures  could  have  some  influence  on 
financial  reporting  or  on  the  management  of 
internal  resources  by  agencies. 

But  this  impression  requires  confirmation. 
There  are  places  to  turn  for  insights  into  the 
practice  of  and  the  potential  for  performance 
measurement.  In  fact,  many  of  the  proponents 
of  the  federal  government's  adoption  of 
performance-based  budgeting  report  that  it 
has  been  successful  in  states,  local  govern¬ 
ments,  and  other  nations.  Understanding 
what  these  efforts  have  achieved,  and  what 
they  have  not,  can  assist  in  evaluating  the  po¬ 
tential  for  performance-based  budgeting  to 
have  a  lasting  effect  on  resource  allocation  in 
the  federal  government.  Moreover,  the  federal 
government  has  adopted  budgeting  systems 
that  were  designed  to  connect  the  effective¬ 
ness  of  federal  programs  to  resource  alloca¬ 
tion.  In  addition,  federal  agencies  have  widely 
varying  experiences  and  abilities  to  measure 
performance.  For  this  reason,  reviewing  past 
and  current  performance  measurement  prac¬ 
tices  can  help  to  discover  what  federal  exper¬ 
ience  and  the  experience  of  other  governments 
suggest  about  the  possibility  of  performance 
measurement  now  in  the  federal  government. 


Chapter  Two 


Linking  Performance  Measurement 
and  Budgeting:  The  State,  Local,  and 
International  Experience 


The  experiences  of  state  and  local  gov¬ 
ernments  have  provided  much  of  the 
impetus  for  a  renewed  call  for  perfor¬ 
mance  measurement  in  the  federal  govern¬ 
ment.  In  addition,  governments  in  other  in¬ 
dustrialized  countries  have  been  trying  to  im¬ 
prove  their  ability  to  measure  performance. 
Proponents  of  changes  in  federal  policy  often 
point  to  the  reported  successes  of  these  other 
governments  in  prescribing  changes.  But 
knowing  whether  other  governments  suc¬ 
ceeded  requires  that  two  questions  be  asked 
about  their  practices.  First,  are  the  measure¬ 
ments  that  they  use  gauges  of  outcome  or  re¬ 
sults,  or  are  they  limited  to  activity  or  pro¬ 
cess?  Second,  to  what  extent  are  these  perfor¬ 
mance  measurement  systems  integrated  into 
the  budget  process?  For  example,  how  clearly 
are  performance  measures  tied  to  choices 
about  resource  allocation,  decisions  about 
agency  management  of  resources,  or  financial 
reporting? 


Experiences  of  Other 
Governments  with 
Performance 
Measurement  and 
Budgeting 

The  general  findings  of  a  review  of  the  perfor¬ 
mance  measurement  practices  of  other  gov¬ 
ernments  indicate  that  although  all  of  the 


units  of  government  studied  use  some  form  of 
performance  measures,  these  measures  con¬ 
tinue  to  be  heavily  focused  on  the  activities  of 
agencies  rather  than  the  results  of  those  ac¬ 
tivities.  Moreover,  performance  measures  are 
only  tenuously  tied  to  governmentwide  re¬ 
source  allocation  decisions.  They  do  benefit 
internal  agency  management  and  financial  re¬ 
porting,  however,  and  provide  important  sig¬ 
nals  about  possible  pitfalls  to  avoid  in  estab¬ 
lishing  a  performance  measurement  system 
for  the  federal  government. 

Several  generalizations  may  be  particularly 
relevant  as  the  federal  government  considers 
embracing  performance  measurement  and 
switching  from  a  budget  system  based  on  in¬ 
puts  to  one  that  is  based  on  performance. 
These  generalizations  address  the  differences 
between  structure  at  different  levels  of  gov¬ 
ernment,  the  prevalence  of  outcome  measures 
in  these  governments,  the  usefulness  of  the 
systems  for  budgeting,  and  the  costs  incurred 
by  other  governments  in  setting  up  perfor¬ 
mance  measurement  systems. 

Relationship  Between 
Government  Structure 
and  Agreement  on  Goals 

Performance  measurement  systems  seem  to 
have  worked  best  for  two  forms  of  government: 
on  the  local  level,  the  council/manager  forms 
of  local  government,  such  as  those  in  Dayton, 
Ohio,  and  Sunnyvale,  California;  and  on  the 
national  level,  governments  with  parliamen- 
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tary  systems,  such  as  New  Zealand’s  and 
Great  Britain’s.  In  both  cases,  the  develop¬ 
ment  of  goals  and  the  setting  of  objectives  cru¬ 
cial  to  the  generation  of  meaningful  measures 
are  encouraged  by  a  concentration  of  political 
power  in  only  one  branch  of  government.  City 
managers  report  directly  to  city  councils,  and 
the  government  in  parliamentary  systems  is 
the  party  in  power.  The  result  is  clear  signals 
from  political  leaders  to  agencies.  In  state 
governments  with  both  a  strong  legislature 
and  strong  governor,  and  in  the  U.S.  national 
government,  the  political  system  has  not  pro¬ 
moted  agreement  on  goals,  particularly  when 
the  branches  are  controlled  by  different  politi¬ 
cal  parties. 

Prevalence  of  Outcome 
Measures 

Outcome  measurements  are  relatively  scarce 
in  the  state,  local,  and  international  govern¬ 
ments  analyzed  by  the  Congressional  Budget 
Office  (CBO).  Measures  of  work  load,  process, 
or  output  are  most  prevalent.  Even  in  juris¬ 
dictions  such  as  Dayton;  Charlotte,  North 
Carolina;  and  Florida  that  have  used  mea¬ 
surement  systems  for  many  years,  only  limit¬ 
ed  progress  has  been  made  in  moving  to  a  sys¬ 
tem  that  focuses  on  outcomes.  The  reason  is 
simple:  most  public  programs  find  outcomes 
very  difficult  to  measure.  The  difficulty  of 
measuring  results  led  New  Zealand's  govern¬ 
ment  to  explicitly  target  outputs  for  measure¬ 
ment,  excluding  outcomes.  This  approach  can 
result  in  significant  advances,  but  uses  of  the 
measures  are  limited.  Citizen  satisfaction 
surveys,  such  as  those  done  in  Dayton  and  in 
Portland,  Oregon,  are  a  notable  exception. 
But  even  those  have  pitfalls  since  they  are  ap¬ 
plicable  to  a  limited  range  of  government  pro¬ 
grams  and  it  is  difficult  to  establish  bench¬ 
marks  for  comparison. 

Use  of  Performance  Measures 
for  Budgeting 

Performance  measures  can  influence  budget¬ 
ing  in  three  ways:  through  performance-based 


budgeting,  management  of  resources  at  the 
agency  level,  or  financial  reporting.  In  state, 
local,  and  other  national  governments,  perfor¬ 
mance  measures  are  generally  used  for  the 
latter  two  purposes  and  not  to  inform  or  influ¬ 
ence  substantially  the  allocation  of  resources. 
The  links  between  outcome  measurement  and 
the  budget  process  are  particularly  weak  in 
many  of  these  jurisdictions. 

In  virtually  all  cases,  budgeting  is  done  ac¬ 
cording  to  a  traditional  incremental  model  (in¬ 
creases  and  decreases  in  agency  budgets  are 
mainly  uniform),  not  based  on  performance. 
The  rationales  for  spending  may  be  formed  on 
the  basis  of  anecdotal  information  about  the 
results  of  programs  but  not  on  hard  evidence. 
In  particular,  governments  that  are  cutting 
spending  tend  to  employ  across-the-board 
methods  for  budget  reductions,  even  though 
performance  measurement  systems  should 
foster  more  careful  targeting  of  budget  reduc¬ 
tions.  State  success  stories  in  performance 
budgeting  are  particularly  difficult  to  find. 
Local  governments  and  national  governments 
outside  the  United  States  are  more  advanced 
than  state  governments  in  their  use  of  perfor¬ 
mance  measures  for  budgeting.  Although  14 
states  report  outcome  measures  in  budget  doc¬ 
uments,  the  ties  of  those  measures  to  budget 
allocation  decisions  are  tenuous. 

Further,  although  General  Accounting  Of¬ 
fice  (GAO)  and  CBO  site  visits  have  uncovered 
some  states  that  have  potential  for  developing 
ties  between  performance  measures  and  bud¬ 
geting,  the  current  efforts  in  these  states  are 
not  far  enough  along  in  order  for  CBO  to  judge 
how  well  they  will  perform.  The  commitment 
of  agencies  is  strongest  when  their  managers 
see  a  benefit  that  accompanies  the  move  to 
performance  measures.  For  example,  ensur¬ 
ing  that  performance  measurement  will  lead 
to  justifying  an  agency’s  budget  in  the  face  of 
fiscal  limits  (as  in  Oregon)  often  results  in  ac¬ 
ceptance  of  the  system  at  that  level. 

Even  if  elected  officials  and  other  policy¬ 
makers  decide  that  linking  performance  mea¬ 
sures  and  budgets  at  a  governmentwide  level 
is  not  advantageous,  individual  agencies  have 
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much  to  gain  from  improving  the  linkage  be¬ 
tween  performance  measures  and  resources. 
In  Florida  and  Oregon,  human-services  agen¬ 
cies  have  used  performance  measures  to  influ¬ 
ence  agency  management.  In  Dayton,  the  Hu¬ 
man  and  Neighborhood  Resources  Depart¬ 
ment  uses  performance  measures  to  target 
funds  toward  individual  parks  and  recreation¬ 
al  facilities.  In  fact,  given  that  elected  officials 
in  these  jurisdictions  have  paid  scant  atten¬ 
tion  to  performance  measures,  it  is  likely  that 
the  real  benefit  results  from  enabling  agencies 
to  target  their  use  of  a  relatively  fixed  set  of 
resources  toward  the  achievement  of  stated 
objectives.  In  short,  performance  measure¬ 
ment  is  much  more  likely  to  be  effective  at  the 
agency  level  than  to  influence  government¬ 
wide  budget  allocations. 

Finally,  performance  measures  will  prob¬ 
ably  be  used  much  more  for  financial  report¬ 
ing  in  the  future.  The  efforts  of  the  Govern¬ 
mental  Accounting  Standards  Board  (GASB) 
have  already  expanded  the  number  of  perfor¬ 
mance  measures  that  are  appended  to  govern¬ 
ment  financial  statements.  The  future  should 
see  not  only  an  expansion  of  the  number  of 
governments  that  report  measurements  in 
their  financial  reports,  but  an  increased  reli¬ 
ance  on  outcome-oriented  measures  as  a  re¬ 
placement  for  the  more  traditional  output 
measures.  Indeed,  within  10  years,  the  GASB 
may  require  that  all  financial  statements  in¬ 
clude  measures  of  service  efforts  and  accom¬ 
plishments  (SEA). 

Costs  and  Benefits  of  Generating 
Performance  Information 

Performance  measurement  systems  can  gen¬ 
erate  a  great  deal  of  information,  but  the  in¬ 
formation  must  be  used  in  order  to  justify  the 
cost.  Setting  up  a  governmentwide  perfor¬ 
mance  measurement  system  can  be  expensive. 
Although  some  jurisdictions  (most  notably, 
Sunnyvale)  use  this  information  for  manage¬ 
ment  and  budgeting  purposes,  others  collect  a 
lot  of  useless  data.  In  most  of  the  state  and  lo¬ 
cal  governments  that  CBO  visited,  questions 
were  raised  concerning  the  usefulness  of  the 


data.  Not  only  can  this  kind  of  data  gathering 
be  wasteful,  but  it  antagonizes  agency  per¬ 
sonnel  whose  support  is  necessary  in  order  for 
such  a  program  to  work.  Therefore,  it  is  im¬ 
portant  to  understand  exactly  how  the  col¬ 
lected  data  are  going  to  be  used  before  man¬ 
dating  the  reporting  of  measures. 


Results  of  Performance 
Measurement  in  Local 
Government 

The  progressive  reform  movement  of  the  early 
20th  century  was  focused,  first  and  foremost, 
on  city  governments.  Professional  city  man¬ 
agers  moved  cities  away  from  purely  political 
administration  by  involving  experts  in  day-to- 
day  management.  Most  of  the  attempts  at 
linking  performance  and  budgeting  in  the 
United  States  have  been  made  at  the  level  of 
local  government. 

In  general,  the  responsibilities  of  local  gov¬ 
ernments  in  the  United  States  make  it  easier 
for  them  to  develop  performance  measures 
than  for  the  state  or  national  governments  to 
do  so.  Because  a  larger  portion  of  their  bud¬ 
gets  is  devoted  to  the  direct  delivery  of  ser¬ 
vices  (police,  fire,  utilities,  schools,  and  so  on), 
performance  is  somewhat  easier  to  measure. 
Moreover,  local  governments  provide  these 
services  to  citizens  who  live  in  a  relatively 
narrow  geographic  area,  making  accountabil¬ 
ity  somewhat  easier.  Particularly  in  coun¬ 
cil/manager  forms  of  local  government,  there 
is  considerably  more  agreement  on  goals,  or 
there  is  a  clear  process  for  adjudicating  dis¬ 
putes.  The  manager  works  for  the  council  in  a 
way  that  has  no  parallel  at  other  levels  of  U.S. 
government  (that  is,  governors  are  not  ap¬ 
pointed  by  state  legislatures,  and  the  Presi¬ 
dent  is  not  appointed  by  the  Congress). 

One  impetus  for  performance  measurement 
in  local  government  has  been  the  move  toward 
common  accounting  standards.  This  effort, 
spearheaded  by  the  GASB's  service  efforts  and 
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accomplishments  reporting,  emphasizes  the 
results  of  government  activity  rather  than  the 
activities  themselves.1  The  GASB  encourages 
local  and  state  governments  subscribing  to  its 
standards  to  improve  their  reporting  of  perfor¬ 
mance. 

The  GASB  has  issued  a  series  of  SEA  re¬ 
ports  covering  a  wide  range  of  state  and  local 
services,  including  education,  transportation, 
public  safety,  public  health,  and  economic  de¬ 
velopment.  These  reports  provide  prototype 
indicators  for  each  of  these  service  areas  and 
recommend  ways  in  which  these  measures  can 
be  used  to  evaluate  performance  for  govern¬ 
ment  programs.  In  particular,  the  reports  ad¬ 
vocate  giving  managers  the  ability  to  report 
explanatory  information  when  actual  perfor¬ 
mance  deviates  from  planned  performance; 
that  ability  is  particularly  important  when 
events  beyond  the  control  of  program  manag¬ 
ers  cause  the  deviation.  Further,  the  reports 
discuss  the  purposes  for  which  SEA  data  can 
be  used,  including  external  reporting,  internal 
management,  and  budget  decisionmaking. 
Although  SEA  reporting  is  catching  on  (par¬ 
ticularly  in  local  government),  one  recent  es¬ 
timate  suggests  that  it  may  take  as  long  as  10 
years  before  standards  actually  take  effect.2 

CBO  visited  several  local  jurisdictions- 
Dayton,  Ohio.;  Charlotte,  North  Carolina;  St. 
Petersburg,  Florida;  and  Portland,  Oregon- 
that  use  performance  measurement  systems. 
In  addition,  the  General  Accounting  Office 
and  the  Office  of  Management  and  Budget  vis¬ 
ited  Sunnyvale,  California,  which  has  a  sys¬ 
tem  that  many  people  consider  to  be  among 
the  most  sophisticated  in  the  country.  All  of 
these  jurisdictions  have  council/manager 
forms  of  government.  Their  experiences  sug¬ 
gest  that  it  is  generally  difficult  to  link  perfor¬ 
mance  measurement  and  budgeting,  and  that 


1.  Harry  P.  Ha  try  and  others,  eda.,  Service  Efforts  and  Ac¬ 
complishments  Reporting:  Its  Time  Has  Come,  An  Over¬ 
view  (Governmental  Accounting  Standards  Board, 
1990). 

2.  See  Paul  Epstein,  "Get  Ready:  The  Time  for  Perfor¬ 
mance  Measurement  is  Really  Coming!”  Public  Admin¬ 
istration  Review,  vol.  52,  no.  5  (September/October  1992), 
pp.  513-519. 


it  is  difficult  to  sustain  such  systems  once  they 
are  established.  Nonetheless,  evidence  sug¬ 
gests  that  individual  managers  find  the  mea¬ 
sures  useful  for  managing  their  agencies  or 
programs. 

Dayton,  Ohio 

Dayton  first  inaugurated  a  performance  mea¬ 
surement  system  two  decades  ago.  The  anchor 
of  the  current  system  is  a  strategic  plan  that 
lists  objectives  and  measures  considered  most 
important  to  the  city  over  the  next  five  years. 
Each  department  is  required  to  identify  mea¬ 
sures  that  address  the  key  objectives  of  the 
strategic  plan.  Dayton's  Office  of  Manage¬ 
ment  and  Budget  reviews  both  agency  budgets 
and  agency  objectives.  The  measures  them¬ 
selves  are  a  combination  of  work  load,  effi¬ 
ciency,  and  outcome.  Dayton  was  also  one  of 
the  first  jurisdictions  to  use  citizen  satisfac¬ 
tion  (survey)  data,  which  are  still  collected 
and  used  today. 

The  system  does  not  seem  to  have  had  any 
substantial  effect  on  the  citywide  allocation  of 
resources.  Staff  in  Dayton's  Office  of  Manage¬ 
ment  and  Budget  and  the  heads  of  two  city  de¬ 
partments  (the  Human  and  Neighborhood  Re¬ 
sources  Department  and  the  Planning  Depart¬ 
ment)  agreed  that  the  system  was  not  used  to 
affect  resource  allocation  at  the  departmental 
level.  Agency  budget  levels  are  fairly  predict¬ 
able  from  year  to  year;  the  council  tends  to 
make  relatively  minor  budget  adjustments  at 
the  margins.  The  system  is  used  by  the  agen¬ 
cies,  however,  to  manage  their  internal  re¬ 
sources  (for  example,  to  decide  whether  to  tar¬ 
get  resources  toward  parks  or  recreation  facili¬ 
ties  concentrating  on  particular  types  of  ac¬ 
tivities). 

Dayton  does  not  plan  to  change  the  system. 
A  recent  evaluation  by  Wright  State  Univer¬ 
sity,  however,  suggested  substantial  dissatis¬ 
faction  with  the  system  amonjr  rank-and-file 
city  employees.  Their  dissatisfaction  stems 
from  a  belief  that  the  system  is  applied  incon¬ 
sistently  (that  is,  it  is  not  taken  seriously  at 
all  levels  of  government),  that  it  has  not  been 
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tied  directly  to  the  city  budget,  that  objectives 
do  not  tie  well  to  job  responsibilities,  and  that 
there  are  too  few  measures  that  assist  in  eval¬ 
uating  the  results  of  city  programs.3 

In  short,  the  system  has  not  substantially 
affected  the  budget  allocation  process  in  Day- 
ton.  The  system  is  not  tied  to  budgeting  (ex¬ 
cept  at  the  agency  level),  but  agencies  such  as 
the  Human  and  Neighborhood  Resources  De¬ 
partment  indicate  that  they  use  the  system  to 
manage  their  own  resources.  Budget  office 
employees  report  that  the  system  (particularly 
the  citizen  satisfaction  surveys)  is  used  as  a 
management  tool  by  the  city  manager.  More¬ 
over,  Dayton  reports  the  results  of  these  sur¬ 
veys  and  other  performance  measures  to  the 
general  public  and  to  elected  officials  as  a  part 
of  its  strategic  planning  process.  With  the  ex¬ 
ception  of  citizen  survey  data,  outcome  mea¬ 
sures  do  not  seem  to  be  prevalent  in  Dayton. 


Charlotte,  North  Carolina 

Charlotte's  system  is  also  20  years  old.  City 
officials  say  that  the  first  performance  mea¬ 
surement  effort  was  unsuccessful,  largely  be¬ 
cause  agency  managers  thought  that  the  data 
collected  were  of  limited  use.  Although  the 
initial  system  focused  almost  exclusively  on 
output  data,  the  use  of  more  qualitative  mea¬ 
sures  has  increased  over  time.  In  addition, 
within  the  last  three  or  four  years,  the  city  has 
tried  to  expand  the  use  of  efficiency  measures. 
The  movement  toward  measuring  program 
outcomes  has  not  progressed  as  far  to  date. 

Performance  data  are  collected  as  a  part  of 
agency  budget  requests,  and  the  budget  office 
plays  a  major  role  in  suggesting  and  refining 
the  measures.  Objectives  are  not  tied  directly 
to  resource  allocation.  Major  changes  in  the 
budget  are  not  driven  by  performance  mea¬ 
sures.  In  practice,  except  for  the  police  depart¬ 
ment  (which  tends  to  be  exempt  from  budget 


3.  Wright  State  University,  Center  foT  Urban  and  Public 
Affairs,  MBO  Evaluation  Report  (Dayton,  Ohio:  Wright 
State  University,  September  3, 1991). 


cuts)  agencies  usually  receive  inflationary  in¬ 
creases  (in  good  economic  times)  or  across-the- 
board  cuts  (in  bad  times). 

Agencies  submit  three  reports  each  year  to 
the  budget  office,  city  manager,  and  council. 
These  reports  focus  on  the  actual  achievement 
of  objectives  compared  with  the  planned  level 
and  are  issued  at  five  months,  eight  months, 
and  the  end  of  the  year.  Most  agencies,  how¬ 
ever,  do  not  use  this  information  for  internal 
management  purposes.  Some,  such  as  the 
city’s  Department  of  Transportation  and  the 
Solid  Waste  Services  Department,  do  use  mea¬ 
sures,  but  it  is  not  clear  whether  the  require¬ 
ment  to  collect  measures  is  a  motivating  fac¬ 
tor.  The  city  council  has  not  used  the  system 
for  budgeting  or  any  other  purpose. 

Although  some  people  have  recommended 
abolishing  the  system,  the  city  is  reluctant  to 
scrap  it  without  a  replacement.  The  budget  of¬ 
fice  has  initiated  such  an  effort  that  would  pay 
more  attention  to  a  hierarchy  of  objectives-a 
movement  that  would  stress  developing  mis¬ 
sion  statements,  strategic  plans,  objectives, 
and  outcome  measures. 


St.  Petersburg,  Florida 

St.  Petersburg  has  experimented  with  perfor¬ 
mance  measurement  since  1973.  The  initial 
effort  was  discontinued  in  the  late  1970s.  St. 
Petersburg's  second  stage  of  performance  mea¬ 
surement  began  in  the  mid-1980s,  when  a 
management  study  recommended  that  a  sys¬ 
tem  be  developed  to  measure  performance 
more  effectively.  The  system  has  three  compo¬ 
nents:  efforts  to  develop  mission  statements, 
goals,  objectives,  and  measures;  surveys  to 
measure  citizens'  satisfaction  with  specific  in¬ 
ternal  services;  and  management  studies  of 
particular  problem  areas. 

Agencies  are  required  to  report  measures  of 
performance  when  they  send  their  annual 
budget  request  to  the  city  manager  and  the 
council.  The  measurements  are  primarily  as¬ 
sessments  of  work  load  and  have  not  been  used 
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much  by  the  manager  and  the  council  in  mak¬ 
ing  budget  decisions.  Further,  of  three  agen¬ 
cies  interviewed— recreation,  police,  and 
planning-only  the  police  department  seemed 
to  be  using  its  measures  for  internal  manage¬ 
ment.  All  three  agencies  agreed  that  the  man¬ 
ager  and  council  did  not  use  them.  In  fact,  al¬ 
though  there  is  some  recognition  that  data 
overload  killed  the  old  system,  the  agencies 
still  perceive  that  they  are  producing  mea¬ 
sures  for  no  good  reason.  The  budget  office  is 
working  to  interest  the  council  and  city  man¬ 
ager  in  the  system  by  including  the  measure¬ 
ments  in  the  briefing  book  that  accompanies 
the  budget  to  the  council.  In  fact,  during  the 
budget  process  for  fiscal  year  1993,  the  inter¬ 
est  in  performance  data  by  both  the  public  and 
the  council  increased  in  response  to  these 
efforts. 


Portland,  Oregon 

Through  the  office  of  the  city  auditor,  the  city 
of  Portland  prepares  an  annual  report  of  city 
service  results,  modeled  after  the  GASB  call 
for  service  efforts  and  accomplishments  re¬ 
porting.  Portland's  SEA  report  focuses  on  the 
largest  and  most  visible  city  services,  such  as 
police,  fire,  parks,  and  street  maintenance. 
The  report  covers  approximately  70  percent  of 
city  spending  and  staff. 

The  auditor’s  office  used  three  general  ap¬ 
proaches  in  developing  the  report.  First,  a 
trend  analysis  compared  data  on  Portland  ser¬ 
vices  to  past  service  levels  and  standards  of 
performance.  Second,  the  results  of  Portland 
city  services  were  compared  with  those  of  six 
other  cities.  Third,  the  auditor’s  office  admin¬ 
istered  a  citizen  satisfaction  survey.  All  large 
city  agencies  participated,  even  though  their 
participation  was  voluntary. 

CBO  interviewed  three  officials  from  agen¬ 
cies  participating  in  the  SEA  program.  Each 
of  these  officials  had  parallel  efforts  to  mea¬ 
sure  outcomes  in  place  even  before  the  SEA 
initiative.  The  SEA  measures,  then,  were 
largely  a  recasting  of  data  that  the  agencies 
were  already  collecting.  The  managers  did  not 


feel  that  the  results  of  the  citizen  surveys  were 
particularly  useful.  In  one  case,  the  managers 
felt  that  the  survey  was  inadequate.  The  oth¬ 
er  two  respondents  could  not  use  the  survey 
results  to  evaluate  agency  performance  be¬ 
cause  they  lacked  baseline  data.  In  late  1992, 
however,  information  reported  to  the  city 
council  began  to  focus  attention  on  problems 
that  the  measures  suggested.  All  respondents 
agreed  that  outcome  measures  should  be  tied 
to  the  budget  process,  but  only  the  parks  and 
recreation  department  has  any  specific  near- 
term  plans  to  do  so.  The  auditor’s  office 
changed  the  timing  of  the  issuance  of  the  1992 
report  in  an  attempt  to  promote  the  use  of  the 
information  in  the  budget  process. 

Although  the  report  summarizes  the  results 
of  city  activities  for  Portland  citizens  and  oth¬ 
er  interested  parties,  agency  practices  seem  to 
have  changed  only  slightly  in  response  to  the 
survey.  In  part,  the  slight  change  can  be  at¬ 
tributed  to  the  emphasis  on  external  report¬ 
ing,  in  addition  to  the  fact  that  city  agencies 
are  already  initiating  performance  measure¬ 
ment  systems.  Particular  progress  seems  to 
have  been  made  in  the  area  of  citizen  surveys, 
but  using  the  surveys  to  inform  management 
or  budgeting  will  depend  on  developing  sev¬ 
eral  years  of  data  for  comparison  purposes.  In 
addition,  the  city  has  had  some  difficulty  in 
collecting  comparable  data  from  other 
jurisdictions. 

Sunnyvale,  California 

Sunnyvale  has  perhaps  the  best-known  sys¬ 
tem  for  measuring  the  performance  of  local 
government.  The  city  of  Sunnyvale  began  to 
develop  a  system  in  the  early  1970s  in  re¬ 
sponse  to  an  effort  spearheaded  by  the  General 
Accounting  Office  to  assist  state  and  local  gov¬ 
ernments  in  developing  their  budgeting  and 
accounting  systems.  The  city  developed  a 
more  elaborate  system  in  the  late  1970s.  It 
combines  strategic  planning,  performance- 
based  budgeting,  and  pay  for  performance  for 
management  employees.  Each  department 
identifies  measurable  performance  goals;  bud¬ 
gets  and  goals  are  closely  tied. 
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In  1991,  the  Office  of  Management  and  Bud¬ 
get  and  GAO  sent  a  joint  delegation  to  visit 
Sunnyvale.  The  delegation  reported  that  the 
city's  budget  system  integrates  performance 
measures,  agency  management,  resource  al¬ 
location  decisions,  and  long-range  planning. 
Several  factors  seem  to  contribute  to  the  suc¬ 
cess  of  the  Sunnyvale  system.  First,  the  city 
operates  on  a  full  cost-accounting  basis,  which 
allows  officials  to  identify  with  some  precision 
how  much  it  costs  to  provide  different  levels  of 
service.  Second,  the  budget  focuses  on  the  out¬ 
puts  that  will  be  produced  rather  than  on 
spending.  In  fact,  the  city  council  does  not 
even  vote  on  the  budget,  in  a  traditional  line- 
item  sense.  Rather,  the  council  approves  goals 
for  city  programs,  and  the  level  of  resources 
necessary  to  meet  those  goals  is  implicit. 
Third,  a  detailed  set  of  objectives  and  perfor¬ 
mance  measurements  govern  all  municipal 
functions.  Many  of  these  measures  are  ori¬ 
ented  toward  outcomes.  Fourth,  early- 
warning  and  long-range  planning  are 
stressed.  The  city  sets  overall  goals  and  objec¬ 
tives  20  years  in  advance  and  projects  the  rev¬ 
enues  and  expenditures  necessary  to  achieve 
these  objectives  for  10  years  in  the  future. 
Fifth,  under  a  detailed  pay-for-performance 
system,  there  are  explicit  ties  between  indi¬ 
vidual  and  organizational  performance.4 

The  OMB/GAO  site  visit  verified  the  gen¬ 
eral  impression  that  the  system  works  well  for 
Sunnyvale,  but  some  skepticism  remains 
about  whether  such  a  system  could  be  adapted 
to  the  federal  government.  The  council/man¬ 
ager  system  in  Sunnyvale  incorporates  a  de¬ 
gree  of  agreement  on  goals  that  may  be  diffi¬ 
cult  to  sustain  in  other  forms  of  government. 
In  addition,  the  expansion  of  Sunnyvale's  tax 
base  in  recent  years  means  that  the  city  has 
operated  without  significant  resource  con¬ 
straints,  which  has  allowed  some  certainty  in 
long-term  resource  planning.  Moreover,  the 
city's  managers  are  held  fully  accountable 


4.  Statement  of  Thomas  F.  Lewcock,  City  Manager  of 
Sunnyvale,  California,  before  the  Senate  Committee  on 
Governmental  Affairs,  May  23, 1991. 


for  results;  salary  bonuses  are  tied  to  the 
achievement  of  planned  outputs  or  outcomes. 


State  Governments’ 
Experiences  with 
Performance 
Measurement 

Many  states  have  generated  performance  data 
in  the  past,  and  these  measurements  have 
generally  not  influenced  resource  allocation. 
Several  states  are  undertaking  a  renewed  ef¬ 
fort  to  carry  out  statewide  management  or 
budgeting  reforms.  Although  state  govern¬ 
ments  do  not  focus  as  intensely  as  local  gov¬ 
ernments  on  direct  service  delivery,  they 
nonetheless  administer  many  programs  that 
should  be  good  candidates  for  outcome  mea¬ 
surement.  Moreover,  the  GASB  service  efforts 
and  accomplishments  requirements  have  af¬ 
fected  state  governments  as  well,  although 
SEA  reporting  has  not  progressed  as  far  in 
state  government  as  it  has  in  some  local  gov¬ 
ernments. 

The  majority  of  states  use  performance 
measures  in  their  budget  processes,  but  these 
measures  usually  do  not  concentrate  on  pro¬ 
gram  effectiveness.  The  National  Association 
of  State  Budget  Officers  recently  surveyed 
performance  management  practices  in  the 
states.  The  survey  found  that  in  1991  34 
states  reported  that  they  used  performance 
measures  as  part  of  their  budget  requests. 
Most  of  these  states  required  that  agencies 
submit  performance  measures  with  their  re¬ 
quests.  Only  14  of  these  states  reported  that 
they  used  measures  of  program  effectiveness 
(outcomes  or  results).  The  measurements  are 
typically  reported  in  the  executive  budget  doc¬ 
ument.  They  usually  cover  three  years  or 
more  and  report  these  data  at  least  at  the  pro¬ 
gram  level. 

State  uses  of  performance  measures  have 
increased.  Another  recent  study  that  reported 
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the  results  of  a  series  of  surveys  of  state  bud¬ 
get  practices  between  1970  and  1990  re¬ 
marked  on  the  use  of  program  performance 
measures  in  budgeting.  5  The  survey  found 
that  almost  all  states  required  agencies  to  sub¬ 
mit  productivity  and  effectiveness  (an  ap¬ 
proximate  surrogate  for  output  and  outcome) 
measures  when  requesting  funds  for  new  pro¬ 
grams  in  1990.  Only  about  one-quarter  of  the 
states  had  required  such  measures  in  1970. 
Moreover,  the  percentage  of  states  reporting 
measures  of  effectiveness  in  the  budget  docu¬ 
ment  increased  from  29  percent  in  1970  to  65 
percent  in  1990.  The  use  of  productivity  mea¬ 
sures  and  projections  has  also  increased  mark¬ 
edly  over  the  20  years. 

There  is  a  difference,  of  course,  between  col¬ 
lecting  and  reporting  measurements  and  us¬ 
ing  them  in  any  significant  way  to  affect  bud¬ 
get  decisions.  To  review  current  practice  and 
future  prospects  for  linking  performance  mea¬ 
surement  and  budgeting,  CBO  visited  two 
states-Florida  and  Oregon-that  are  consid¬ 
ered  to  be  among  the  leaders  in  developing 
performance  measurement  systems.  In  addi¬ 
tion,  CBO  reviewed  the  results  of  a  recent 
GAO  study  that  included  data  from  several 
other  states,  including  Hawaii,  Iowa,  North 
Carolina,  Louisiana,  and  Connecticut.  Both 
the  states  visited  by  CBO  and  those  in  the 
GAO  study  were  chosen  specifically  on  the  ba¬ 
sis  of  reports  that  they  had  substantial  perfor¬ 
mance  measurement  efforts  in  process. 


Florida 

The  state  of  Florida  has  been  attempting  to  tie 
performance  measurement  to  the  budget  for 
25  years.  Over  that  time,  each  agency  has 
been  required  to  report  measurements  with 
the  budget  request  that  goes  concurrently  to 
the  governor  and  the  legislature.  State  offi¬ 
cials  interviewed  by  CBO  uniformly  viewed 


5.  Robert  D.  Lee,  Jr.,  "Use  of  Program  Information  and 
Analysis  in  State  Budgeting  Trends  of  Two  Decades,”  in 
Thomas  Lynch  and  Lawrence  Martin,  eds.,  Handbook  of 
Comparative  Public  Budgeting  and  Financial  Manage¬ 
ment  (New  York:  Marcel  Dekker,  1991). 


this  system  as  creating  a  lot  of  paper  without 
significantly  influencing  budget  policy.  This 
is  mainly  because  the  measurements  have 
been  focused  on  output  in  the  past  and  have 
not  been  very  useful  for  setting  policy. 

Recently,  however,  there  has  been  renewed 
interest  in  performance  measurement,  which 
will  be  included  in  the  current  statewide  plan¬ 
ning  process.  At  the  highest  level,  a  series  of 
Florida  benchmarks  will  include  a  series  of 
100  quality-of-life  indicators,  which  are  in¬ 
tended  to  serve  as  guidelines  for  policymak¬ 
ing.  Part  of  this  process  involves  the  develop¬ 
ment  of  agency  plans  related  to  state  planning 
objectives. 

Although  the  linkage  of  performance  mea¬ 
sures  to  resource  allocation  statewide  is  not 
likely  to  occur  in  the  immediate  future,  it 
could  happen  earlier  at  the  individual  agency 
level.  For  example,  the  Department  of  Health 
and  Rehabilitative  Services  (HRS),  which  is 
the  umbrella  agency  for  human  services,  is  at¬ 
tempting  to  develop  agency  plans  that  feed  in¬ 
to  the  statewide  plan  and  to  include  outcome 
measures  in  this  plan. 

Florida  planners  hope  to  tie  the  statewide 
planning  process  and  the  budget  process  to¬ 
gether.  They  would  develop  agency  objectives 
that  tie  into  the  statewide  plan,  and  the  legis¬ 
lature  would  give  lump-sum  appropriations  to 
those  agencies  and  hold  them  accountable  for 
results.  These  efforts  would  change  the  bud¬ 
get  process  from  a  system  in  which  controlling 
expenditures  is  paramount  to  one  in  which 
planning  for  the  future  is  perceived  as  the 
most  important  goal. 

Florida  is  also  trying  to  refine  the  process  of 
developing  agency  plans  (similar  to  the  HRS 
plan)  and  induce  agencies  to  develop  better 
measurements.  Within  one  year,  Florida 
would  require  all  agencies  to  measure  out¬ 
comes.  Eventually,  the  state  hopes  to  tie  in¬ 
dividual  agency  indicators  to  the  state  plan 
and  to  report  those  indicators  in  its  budget. 

Over  the  years,  Florida’s  budget  process  has 
required  agencies  to  produce  a  lot  of  perfor- 
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mance  data,  but  the  state's  failure  to  tie  per¬ 
formance  measures  to  the  budget  has  antago¬ 
nized  many  agencies.  Furthermore,  a  discus¬ 
sion  with  the  state  auditor's  office  indicated 
some  problems  with  the  reliability  of  data.  In 
some  cases,  audits  were  unable  to  verify  the 
existence  or  accuracy  of  the  performance  data 
being  reported.  Planning  and  budgeting  pro¬ 
cesses  are  not  linked,  although  this  is  an  ex¬ 
plicit  goal  for  the  future.  Even  if  the  system  is 
never  used  for  budgeting  or  gubernatorial 
management,  however,  the  efforts  of  individ¬ 
ual  agencies  produce  some  benefits.  For  ex¬ 
ample,  if  HRS  manages  its  resources  better 
and  more  in  line  with  stated  goals,  this  effect 
is  positive  regardless  of  whether  it  influences 
the  state  legislature's  or  governor's  decisions 
about  resource  allocation. 


Oregon 

Several  events  have  taken  place  concurrently 
to  improve  the  environment  for  performance 
measurement  in  Oregon.  First,  the  state  is  en¬ 
gaged  in  an  ambitious  effort  to  increase  the 
use  of  performance  measures  in  the  budget 
process.  Second,  in  November  1990,  Oregon 
voters  passed  a  local  property  tax  limitation, 
called  Ballot  Measure  5,  which  will  ultimately 
create  a  state  budget  shortfall  of  more  than  15 
percent.6  The  state  also  has  a  long-range 
planning  process,  called  Oregon  Benchmarks, 
which  was  developed  in  advance  of  the  perfor¬ 
mance  measurement  effort. 

Current  initiatives  primarily  focus  on  train¬ 
ing  agencies  to  use  performance  measures 
that  allow  for  the  weighting  of  both  efficiency 
and  effectiveness.  Because  the  budgetary  en¬ 
vironment  after  Ballot  Measure  5  stands  to  be 
much  more  competitive,  agencies  have  incen¬ 
tives  to  develop  measurements,  and  agencies 
that  fail  to  do  so  may  find  themselves  at  a  dis¬ 
advantage  in  the  appropriation  process.  More 
broadly,  one  of  the  governor's  explicit  goals  is 


6.  Ballot  Measure  5  is  expected  to  have  this  effect  on  state 
finance  because  it  would  require  the  state  to  replace  a 
portion  of  property  tax  revenue  lost  by  school  districts. 


to  educate  Oregon  citizens  about  the  services 
provided  by  state  government  and  how  the 
measurement  system  feeds  into  that  effort. 
All  agencies  have  now  participated  in  the 
training  sessions. 

Oregon  is  attempting  to  expand  the  report¬ 
ing  of  performance  measures  in  the  budget 
and  to  link  together  performance  measures 
and  the  benchmarks  in  the  budget  process. 
For  the  last  biennial  budget,  each  agency  was 
required  to  develop  eight  to  ten  indicators  that 
will  be  expanded  and  refined  for  the  following 
two  years.  In  addition,  each  agency  has  been 
given  incentives  to  develop  new  initiatives 
that  tie  in  with  the  Oregon  benchmarks.  For 
the  most  recent  budget  process,  the  state  used 
a  modified  zero-based  budget  process  in  which 
agencies  were  required  to  forecast  the  impact 
of  reducing  their  budgets  by  10  percent  to  20 
percent  from  the  previous  year's  level.  Agen¬ 
cies  were  then  given  opportunities  to  recover  a 
portion  of  these  lost  funds,  in  part  by  adding 
programs  that  are  explicitly  related  to  the  Or¬ 
egon  benchmarks. 

It  is  still  too  early  to  tell  whether  the  perfor¬ 
mance  measurement  effort  has  created  any 
lasting  change  in  Oregon’s  managerial  or  bud¬ 
getary  environment,  but  there  are  some  im¬ 
pressive  signs  of  progress.  Some  agencies 
(particularly  the  Adult  and  Family  Services 
Division)  seem  to  be  using  the  reports  gener¬ 
ated  by  the  measurement  system  as  their  pri¬ 
mary  management  reports.  In  addition,  some 
of  the  measurements  themselves  are  quite  am¬ 
bitious.  For  example,  the  state's  vocational  re¬ 
habilitation  department,  which  had  pre¬ 
viously  collected  little  but  work  load  data,  now 
has  some  very  good  outcome  measures,  includ¬ 
ing  (most  notably)  the  percentage  of  clients 
that  remain  employed  18  months  after  they 
have  left  one  of  the  department's  programs. 
Other  agencies  do  not  seem  to  be  quite  as  com¬ 
mitted  to  the  system. 

Although  there  has  clearly  been  progress  in 
Oregon,  the  picture  is  not  all  positive.  First, 
because  agencies  have  agreed  to  use  the  mea¬ 
surement  process  primarily  as  a  tool  for  sell¬ 
ing  their  programs,  there  are  no  incentives  for 
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reporting  negative  performance.  Second,  the 
use  of  the  system  varies  widely  from  agency  to 
agency;  the  most  important  single  variable 
seems  to  be  the  commitment  of  the  agency 
head.  Third,  though  the  legislature  seems 
supportive,  it  remains  to  be  seen  whether  per¬ 
formance  measures  can  be  used  to  affect  re¬ 
source  allocation  in  any  meaningful  way. 

General  Accounting  Office 
Study  of  Five  States 

The  General  Accounting  Office  has  reported 
the  results  of  a  recent  study  of  five  other 
states-Connecticut,  Hawaii,  Iowa,  Louisiana, 
and  North  Carolina-that  researchers  visited 
to  determine  how  extensively  performance 
measures  were  tied  to  budgeting  in  these 
states.?  GAO  chose  these  states  because  sur¬ 
veys  and  other  reports  had  listed  them  as 
among  national  leaders  in  developing  perfor¬ 
mance  measures.  GAO  researchers  inter¬ 
viewed  both  executive  branch  and  legislative 
branch  officials.  They  reported  that: 

Despite  long-standing  efforts  in 
states  regarded  as  leaders  in  perfor¬ 
mance  budgeting,  performance  mea¬ 
sures  have  not  attained  sufficient 
credibility  to  influence  resource  alloca¬ 
tion  decisions.  Instead,  according  to 
most  of  the  state  legislative  and  execu¬ 
tive  branch  officials  we  interviewed, 
resource  allocation  decisions  continue 
to  be  driven,  for  the  most  part,  by  tradi¬ 
tional  budgeting  practices. . . .  Outside 
the  budget  process,  state  officials  say 
that  performance  measures  have  aided 
managers  in  (1)  establishing  program 
priorities,  (2)  strengthening  manage¬ 
ment  improvement  efforts,  (3)  dealing 
with  the  results  of  budgetary  reduc¬ 
tions,  and  (4)  gaining  more  flexibility 
in  allocating  appropriated  funds. 8 


7.  General  Accounting  Office,  Performance  Budgeting: 
State  Experiences  and  Implications  for  the  Federal  Gov¬ 
ernment,  GAO/AFMD-93-41  (February  1993). 

8.  Ibid.,  p.  1. 


The  results  of  these  site  visits  by  GAO  are 
generally  consistent  with  CBO's  conclusions 
about  Florida  and  Oregon.  Performance  mea¬ 
sures  do  not  influence  governmentwide  deci¬ 
sions  about  resource  allocation  in  these  five 
states;  rather,  more  traditional  budget  prac¬ 
tices  predominate.  Although  budgeting  was 
not  tied  to  performance  measurement  in  any 
of  the  five  states,  state  officials  reported  using 
performance  measures  to  support  manage¬ 
ment  improvement  efforts  within  depart¬ 
ments.  In  short,  if  performance  measures 
make  a  difference  in  these  states,  it  is  in  agen¬ 
cy  use  of  resources  rather  than  in  the  alloca¬ 
tion  of  resources  by  governors  or  state  legisla¬ 
tors. 


Experiences  of  Other 
Nations 

At  least  five  industrialized  countries-Great 
Britain,  Australia,  New  Zealand,  Canada,  and 
Sweden-have  put  into  practice  management 
or  budgeting  reforms  that  have  included  an 
element  of  performance  measurement.  Be¬ 
cause  these  countries  have  parliamentary 
forms  of  government,  it  is  undoubtedly  easier 
to  adopt  and  carry  out  these  reforms  than  it 
would  be  in  the  United  States.  In  such  a  sys¬ 
tem,  the  party  (or  coalition)  in  power  is,  by 
definition,  the  government.  This  contrasts 
with  the  U.S.  system,  in  which  the  Congress 
and  the  Presidency,  even  when  they  are  con¬ 
trolled  by  the  same  party,  can  become  dead¬ 
locked  over  policies.  Still,  the  lessons  of  these 
countries  can  inform  the  development  of  a 
measurement  system  in  the  United  States.9 
These  lessons  generally  focus  on  two  areas: 
the  progress  that  has  been  made  in  improving 
the  measurement  of  results,  and  ties  that  have 
been  developed  between  measurement  and  the 


9.  Unless  otherwise  noted,  the  conclusions  concerning  the 
developments  in  these  countries  are  based  on  Allen 
Schick,  "Budgeting  for  Results:  Recent  Developments  in 
Five  Industrialized  Countries,"  Public  Administration 
Review,  vol.  50,  no.  1  (1990),  pp.  26-34. 
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budgeting  and  financial  management  pro¬ 
cesses. 


Measuring  Results 

Each  of  these  five  countries  continues  to  de¬ 
velop  better  measures,  with  differing  levels  of 
success.  In  many  cases,  the  performance  mea¬ 
surement  efforts  in  these  countries  have  ac¬ 
companied  a  transfer  of  responsibility  from 
the  central  government  to  the  line  agencies, 
which  commit  themselves  to  concrete  perfor¬ 
mance  targets  in  exchange  for  control  of  a 
specified  level  of  resources.  In  Great  Britain, 
experience  thus  far  suggests  that  the  objec¬ 
tives  have  not  been  expressed  precisely 
enough  to  allow  assessment  of  their  achieve¬ 
ment.  Australia's  progress  in  developing  in¬ 
dicators  has  been  confined  mainly  to  the  more 
easily  measurable  efficiency  and  work  load 
targets.  Improving  measurement  continues  to 
be  an  important  objective  in  Australia. 

The  movement  toward  performance  mea¬ 
surement  in  New  Zealand  is  part  of  a  broader 
effort  to  reform  the  public  budgeting  and  ac¬ 
counting  systems.10  First,  the  government  of 
New  Zealand  switched  the  budgeting  system 
from  an  emphasis  on  inputs  to  an  emphasis  on 
outputs.  Second,  the  need  to  assess  the  perfor¬ 
mance  of  all  departments  gave  rise  to  a  move 
from  cash  to  accrual  accounting.11  These 
changes  were  accomplished  through  the  pas¬ 
sage  of  the  Public  Finance  Act  1989. 12  Under 
the  system  established  by  this  act,  the  govern¬ 
ment  specifies  its  outcomes  (objectives),  and 
individual  agencies  are  responsible  for  design¬ 
ing  programs  around  outputs  to  achieve  these 
goals. 


10.  For  a  description  of  the  New  Zealand  system,  see  Fran¬ 
ces  Goldman  and  Edith  Brashares,  "Performance  and 
Accountability:  Budget  Reform  in  New  Zealand,"  Public 
Budgeting  and  Finance,  vol.  11,  no.  4  (1991),  pp.  75-85. 

11.  Accrual  accounting  recognizes  costs  of  decisions  when 
spending  is  committed  to,  rather  than  when  cash  flows 
out  of  the  Treasury. 


Budget  Reform 

In  each  of  these  jurisdictions,  the  relationship 
between  performance  measures  and  the  bud¬ 
get  is  tenuous,  although  in  many  cases  (par¬ 
ticularly  Great  Britain  and  Australia)  the  fo¬ 
cus  of  budgeting  shows  some  signs  of  shifting 
from  measuring  work  load  to  measuring  re¬ 
sults.  In  Great  Britain,  the  linkage  of  perfor¬ 
mance  measures  and  budgets  has  been  pro¬ 
moted  by  the  publication  of  these  measures  in 
the  annual  Public  Expenditure  White  Paper; 
the  paper  for  1988/1989-1990/1991  included 
more  than  1,800  measures. 

Canada  and  New  Zealand  are  exceptions  to 
this  trend,  for  different  reasons.  In  Canada, 
the  performance  measurement  system  has 
made  no  inroads  into  the  budget  process.  New 
Zealand  has  made  progress,  but  only  in  the  ar¬ 
ea  of  measuring  activities  or  outputs;  the  bud¬ 
get  process  ignores  outcomes  at  the  agency 
level.  Agencies  in  New  Zealand  are  only  held 
accountable  for  outputs;  it  is  the  responsibility 
of  the  government  (the  party  that  controls  the 
Parliament)  to  understand  how  outputs  are  re¬ 
lated  to  outcomes.  Unlike  Britain  and  Austra¬ 
lia,  there  is  an  explicit  tie  between  measures 
and  resources  in  New  Zealand,  but  only  at  the 
level  of  outputs,  not  outcomes. 

Finally,  the  Swedish  performance  measure¬ 
ment  effort  is  linked  to  the  budget  through  a 
three-year  budgeting  system  in  which  agen¬ 
cies  are  given  greater  flexibility  to  use  funds 
in  exchange  for  more  information  on  perfor¬ 
mance.  Agencies  have  been  given  the  oppor¬ 
tunity  to  receive  three-year  budgets  in  ex¬ 
change  for  a  more  detailed  budget  justification 
process.  Each  agency  that  participates  in  the 
system  must  submit  an  annual  outcomes  re¬ 
port  in  the  two  intervening  years  of  the  three- 
year  period.  At  the  end  of  the  three-year  cycle, 
the  agency  undergoes  a  new,  thorough  justifi¬ 
cation  process. 


12.  For  an  example  of  financial  reporting  under  the  accrual 
accounting  requirements  of  the  Public  Finance  Act  1989, 
see  Financial  Statements  of  the  Government  of  New  Zea¬ 
land  for  the  Year  Ended  30  June  1992  (October  1992). 


Chapter  Three 


Past  Federal  Efforts 
to  Link  Performance  Measures 
and  Budgeting 


The  current  effort  to  link  performance 
measurement  to  budgeting  more  effec¬ 
tively  is  a  logical  successor  to  three 
other  efforts  by  the  federal  government  in  the 
past  40  years  to  tie  performance  to  budgeting 
on  a  governmentwide  scale:  performance  bud¬ 
geting,  program  budgeting  (including  primar¬ 
ily  the  planning-programming-budgeting  sys¬ 
tem,  also  known  as  PPBS),  and  zero-based 
budgeting  (ZBB).  These  systems  had  one  ba¬ 
sic  characteristic  in  common.  They  repre¬ 
sented,  in  part,  an  effort  to  increase  the  level 
of  information  available  for  federal  budgeting 
and  management.  At  least  two  of  them 
(PPBS  and  ZBB)  attempted  to  budget  on  the 
basis  of  program  effectiveness.  In  general, 
these  systems  achieved  less  than  their  design¬ 
ers  envisioned.  In  particular,  the  effort  that 
has  gone  into  these  systems  did  not  change 
the  way  that  federal  resources  are  allocated. 


Performance  Budgeting 

The  first  Hoover  Commission  (1949)  proposed 
that  more  data  be  integrated  into  the  budget 
process.1  This  proposal,  which  was  referred  to 
as  "performance  budgeting,"  was  designed  to 
allow  managers  to  develop  measures  of  work 


1.  Commission  on  Organization  of  the  Executive  Branch  of 
the  Government,  Budgeting  and  Accounting  ( 1949). 


load  and  cost-effectiveness.  Performance  bud¬ 
geting  was  clearly  aimed  at  increasing  effi¬ 
ciency.  The  objectives  of  programs  were  taken 
as  a  given,  and  the  purpose  of  performance 
budgeting  was  to  determine  the  least  costly 
method  of  achieving  them. 

Performance  budgeting  indicated  a  shift 
from  budgeting  based  on  expenditure  "con¬ 
trol"  (as  was  the  case  with  the  early  line-item 
budgets,  for  example)  to  budgeting  based  in¬ 
creasingly  on  "management"  concerns.  The 
emphasis  was  not  on  making  governmentwide 
budgetary  trade-offs;  rather,  it  was  on  mea¬ 
suring  work  load  on  an  agency  basis.  The  fo¬ 
cus  was  on  the  work  to  be  done,  not  on  the  use¬ 
fulness  of  the  objectives  themselves. 2  Al¬ 
though  performance  budgeting  was  never 
adopted  as  a  governmentwide  budgetary  pro¬ 
cess,  it  is  significant  because  it  emphasized 
the  integration  of  program  information  and 
budgeting.  This  emphasis  was  to  be  continued 
in  future  reform  efforts. 


Program  Budgeting 

The  second  type  of  broad-based  reform,  pro¬ 
gram  budgeting,  had  its  federal  heyday  dur- 


2.  Allen  Schick,  "The  Road  to  PPB:  The  Stages  of  Budget 
Reform,"  Public  Administration  Review,  vol.  26,  no.  4 
(1966),  pp.  29-30. 
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ing  the  administration  of  President  Lyndon 
Johnson.  By  contrast  with  performance  bud¬ 
geting,  program  budgeting  was  explicitly  fo¬ 
cused  on  budgetary  choices  among  competing 
activities.  Although  performance  budgeting  is 
designed  to  discover  which  of  several  ways  is 
the  most  efficient  method  of  accomplishing  a 
given  objective,  program  budgeting  treated 
the  objectives  themselves  as  variable.  As 
such,  program  budgeting  is  not  a  management 
system,  but  a  resource  allocation  system.  It  is 
a  specific  alternative  to  the  traditional  gov¬ 
ernment  budgeting  system,  which  focuses  on 
marginal  adjustments  to  the  status  quo. 

Although  program  budgeting  has  been  at¬ 
tempted  at  various  levels  of  government,  the 
federal  effort  is  embodied  primarily  in  the 
mid-1960s  effort  by  President  Johnson  to  put 
into  effect  a  planning-programming-budget¬ 
ing  system  throughout  the  federal  govern¬ 
ment.  PPBS  was  based  on  a  successful  effort 
by  Secretary  of  Defense  Robert  McNamara  in 
the  19608  to  put  PPBS  into  practice  in  the  De¬ 
fense  Department. 

President  Johnson  ordered  the  Defense  De¬ 
partment's  system  to  be  employed  throughout 
the  federal  government  in  1965.  By  1971, 
however,  requirements  that  agencies  include 
the  PPBS  system  data  were  suspended,  and 
PPBS  ceased  to  exist,  at  least  explicitly,  in  the 
federal  government. 
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o  It  flew  in  the  face  of  existing  budgetary 
traditions  and  relationships;  in  particular, 
many  people  strongly  objected  to  the  sug¬ 
gestion  that  the  budget  process,  which  is 
inherently  political,  could  be  made  "ra¬ 
tional." 

o  It  was  not  given  adequate  resources,  and 
top  managers  were  not  entirely  committed 
to  it. 

o  Good  analysts  and  data  were  in  short  sup¬ 
ply,  and  they  were  necessary  to  produce 
the  kind  of  information  crucial  to  the  suc¬ 
cess  of  PPBS. 

o  Because  PPBS  required  a  review  of  all  ac¬ 
tivities  in  each  year,  the  reform  caused  so 
much  conflict  that  the  political  system  was 
incapable  of  handling  it. 

o  PPBS  made  a  clear  assumption  that  effi¬ 
ciency  was  the  primary  value  to  be  consid¬ 
ered  in  evaluating  the  usefulness  of  pro¬ 
grams,  which  was  not  generally  agreed  to 
and  is  a  difficult  assumption  to  make  in 
the  case  of  a  public  program. 

o  PPBS  was  an  executive  budget  system, 
and  largely  ignored  the  role  of  the  Con¬ 
gress  in  the  budget  process.  The  result 
was  that  the  Congress  ignored  the  system 
in  favor  of  its  established  procedures. 


Reports  and  evaluations  indicate  that  the 
PPBS  was  much  less  successful  in  civilian 
agencies  than  in  the  Defense  Department,  in 
part  because  it  lacked  administrative  commit¬ 
ment  to  integrating  it  fully  into  civilian  agen¬ 
cy  management  systems.  Although  Secretary 
McNamara  clearly  was  committed  to  putting 
PPBS  into  effect  in  the  Defense  Department, 
in  most  other  cases  the  system  was  treated  as 
an  annual  reporting  requirement  rather  than 
as  a  management  or  budgeting  opportunity. 
Several  other  arguments  have  been  made  for 
the  failure  of  PPBS  to  catch  on;3 

o  It  was  introduced  across  the  board  without 
much  preparation. 


PPBS  was  unable  to  fulfill  its  promise  as  a 
budgetary  reform.  In  particular,  it  did  not 
change  the  budget  process  from  one  that  fo¬ 
cused  primarily  on  making  small  incremental 
changes  to  one  that  could  make  comprehen¬ 
sive  alterations.  Like  performance  budgeting, 
however,  PPBS  increased  the  role  of  analysis 
in  budgeting  and  led  to  better-informed  bud¬ 
get  decisions.  This  increase  has  continued 
long  after  PPBS  has  faded  from  view. 


3.  Unless  otherwise  noted,  each  of  these  arguments  is 
taken  from  Allen  Schick,  "A  Death  in  the  Bureaucracy: 
The  Demise  of  Federal  PPB,"  Public  Administration 
Review ,  vol.  33,  no.  2  (March/ April  1973),  pp.  146-156. 
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Zero-Based  Budgeting 

The  Carter  Administration  brought  its  own 
model  of  budget  reform  to  Washington  in  1976 
in  the  form  of  zero-based  budgeting.  ZBB, 
which  President  Carter  had  used  as  governor 
in  Georgia,  was  designed  to  permit  compre¬ 
hensive  review  of  the  federal  budget,  as  op¬ 
posed  to  the  traditional,  more  incremental  ap¬ 
proach. 

In  a  "pure"  ZBB  system,  instead  of  concen¬ 
trating  on  budgetary  changes  at  the  margin, 
all  programs  are  evaluated  each  year.  The 
process  of  arriving  at  a  budget  is  literally  to 
start  from  scratch.  At  the  federal  level,  that 
would  require  answering  such  questions  as, 
"What  if  we  didn’t  have  an  army  and  navy?" 
or  "What  if  Social  Security  did  not  exist?" 

In  practice,  no  government  ZBB  system,  in¬ 
cluding  the  federal  one,  went  this  far.  Presi¬ 
dent  Carter's  Office  of  Management  and  Bud¬ 
get  used  a  variant  of  ZBB  in  which  agencies 
were  asked  to  rank  their  programs  within 
some  funding  limits.  The  question  thus  be¬ 
came,  "What  if  the  Department  of  Defense  on¬ 
ly  received  90  percent  (for  example)  of  the  cur¬ 
rent  year's  funding?"  This  approach,  in  the¬ 
ory,  forces  the  agency  to  assign  priorities  to  its 
programs  in  order  to  decide  what  is  in  the 
"base"  (the  90  percent,  in  this  case)  and  what 
is  to  be  identified  as  a  possible  reduction. 

In  fact,  the  federal  version  of  ZBB  that  was 
practiced  in  the  late  1970s  was  very  similar  to 
the  latter  kind  of  system.  Approximately 
10,000  "decision  packages"  were  prepared 
each  year.  Each  package  included  three  dif¬ 
ferent  funding  levels-a  reduction  level,  a  cur¬ 
rent-services  (baseline)  level,  and  an  enhance¬ 
ment  level  that  offered  upgraded  services.  De¬ 
cision  packages  were  to  be  ranked  by  impor¬ 
tance.  In  practice,  the  reduction  level  as¬ 
sumed  a  minimum  funding  level  of  80  percent 
to  85  percent,  and  budgetary  decisions  con¬ 
centrated  on  the  proposed  additions  to  that 
level.  This  meant  that,  rather  than  starting 
from  a  zero  base,  ZBB  concentrated  on  mar¬ 


ginal  additions  to  an  established  (albeit  lower) 
base,  which  is  also  how  traditional  budgeting 
works. 

In  addition  to  not  being  zero  based,  federal 
ZBB  was  criticized  as  being  excessively  time 
consuming  and  unrealistic.  For  example, 
many  programs  were  bound  by  statute  to  pro¬ 
vide  a  given  level  of  service;  administrators  of 
these  programs  were  hard  pressed  to  identify 
realistic  ways  to  reduce  funding  levels.  ZBB, 
in  practice,  eliminated  or  reduced  very  few 
programs.  President  Reagan  abandoned  the 
experiment  in  1981.4 


Common  Themes 
of  Budgets  and 
Management  Reform 

Each  of  these  reforms  represents  an  attempt 
by  a  President  to  institute  a  "top-down"  solu¬ 
tion  to  federal  budgeting  and  management. 
The  reforms  can  be  lumped  together  into  "ef¬ 
fectiveness  budgeting,"  which  is  characterized 
by  an  effort  to  define  important  objectives  and 
tie  their  achievement  to  the  means  of  attain¬ 
ing  them.  While  each  of  the  reforms  discussed 
in  this  chapter  traveled  a  different  road  to  get 
there,  each  (with  the  exception  of  the  earliest 
reform,  performance  budgeting)  was  con¬ 
cerned  with  linking  the  use  of  resources  and 
the  outcomes  that  result  from  their  use.5  In 
addition,  with  the  exception  of  performance 
budgeting,  each  was  focused  specifically  on 
agency  program  or  budget  outcomes.  Analy¬ 
ses  of  the  various  reforms  and  comparisons 
among  them  suggest  some  common  themes. 


4.  Robert  D.  Lee  and  Ronald  Johnson,  Public  Budgeting 
Systems  (Rockville,  Md.:  Aspen  Publishers,  1989),  p.  86. 

5.  The  term  "effectiveness  budgeting”  is  taken  from  James 
M.  Harkin,  "Effectiveness  Budgeting:  The  Limits  of 
Budget  Reform,"  Policy  Studies  Review,  vol.  2,  no.  3  (Au¬ 
gust  1982),  pp.  112-126. 
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These  " rational "  budgeting  systems  gener¬ 
ally  proved  antithetical  to  those  who  had  some 
stake  in  the  status  quo  process.  Some  critics  of 
rational  budgeting  systems  argue  that  they 
fly  in  the  face  of  political  norms  and  therefore 
cannot  be  made  to  work  in  a  pluralist  society. 
According  to  this  argument,  since  budgeting  is 
about  allocating  resources,  it  is  inherently  a 
political  process.  If  that  is  true,  the  notion 
that  it  is  possible  to  arrive  at  a  rational  alloca¬ 
tion  of  resources  that  is  somehow  "above"  poli¬ 
tics  is  unrealistic.  Furthermore,  the  political 
system  deals  best  with  budgetary  schemes 
that  reduce,  not  emphasize,  conflict. 

Traditional  budgeting  has  the  virtue  of  forc¬ 
ing  fewer  choices  than  rational  budgeting.  It 
does  this  by  concentrating  on  a  minimum 
number  of  decisions,  rather  than  opening  the 
entire  budget  to  reexamination,  and  by  re¬ 
quiring  few  explicit  decisions  to  be  made  about 
how  much  to  give  to  one  group  and  take  away 
from  another.  A  large  portion  of  the  budget  is 
not  subject  to  detailed  annual  review .6  Con¬ 
versely,  proponents  of  these  reform  systems 
argue  that  it  may  be  desirable  to  make  politi¬ 
cally  expedient  decisions  more  difficult.  These 
politically  expedient  decisions  are  usually  ex¬ 
emplified  by  the  political  system’s  paying  ex¬ 
cessive  attention  to  local  concerns  in  allocat¬ 
ing  funds. 

Etich  of  these  systems,  though  ostensibly  ei¬ 
ther  "top-down'*  or  consensual  in  design,  suf¬ 
fered  from  the  lack  of  clear  agreement  between 
superiors  and  subordinates  over  goals  and  ob¬ 
jectives.  Objectives  are  very  hard  to  determine 
and  measure  in  the  public  sector.  In  order  to 
budget  on  the  basis  of  performance,  one  must 
have  an  objective.  Different  actors  in  an  agen¬ 
cy's  environment,  such  as  the  President,  the 
Congress,  interest  groups,  and  agency  man¬ 
agers,  may  have  different  perceptions  of  the 
agency's  goals  or  how  they  should  be  accom¬ 
plished. 


6.  Aaron  Wildavsky,  "A  Budget  for  All  Seasons?  Why 
the  Traditional  Budget  Lasts,"  Public  Administration 
Review,  vol.  38,  no.  6  (November/December  1978),  pp. 
501-509. 


Each  of  the  budgeting  systems  reviewed 
suffered  from  an  inability  to  achieve  agree¬ 
ment  between  these  actors  on  goals  and  objec¬ 
tives.  Support  by  top  managers  and  the  White 
House  must  be  continuous-it  is  not  enough  to 
announce  the  reform  and  then  turn  it  over  to 
agencies  for  carrying  out.  Lacking  that  agree¬ 
ment,  it  is  difficult  to  measure  performance, 
much  less  budget,  on  the  basis  of  it. 

"Top-down"  systems  run  the  risk  of  running 
into  a  hostile  bureaucracy.  It  is  necessary  to  in¬ 
volve  the  agency  staff,  whose  support  will  be 
needed  in  order  to  carry  out  the  program.  In 
addition  to  the  difficulties  of  achieving  consen¬ 
sus  on  goals  and  objectives,  "top-down"  sys¬ 
tems  often  prove  threatening  to  agency  man¬ 
agement.  If  staff  members  believe  that  a  sys¬ 
tem  is  being  established  to  "punish"  nonper¬ 
formance  (by  cutting  budgets,  for  example),  it 
may  be  difficult  to  get  their  cooperation.  That 
may  lead  to  the  development  of  self-serving  in¬ 
dicators,  falsification  of  results,  or  other  coun¬ 
terproductive  actions  on  the  part  of  staff  ad¬ 
ministering  the  program. 

Each  of  these  systems  had  an  almost  over¬ 
whelming  need  for  data  in  order  to  survive. 
With  these  systems,  particularly  PPBS  and 
ZBB,  the  necessity  of  producing  yearly  the  da¬ 
ta  and  paper  justifying  or  analyzing  a  full 
range  of  government  programs  was  more  than 
the  system  could  handle.  Indeed,  many  ana¬ 
lysts  would  argue  that  PPBS  and  ZBB  died  of 
their  own  weight.  A  process  that  produces 
mounds  of  paper  that  is  neither  read  nor  used 
for  decisionmaking  is  inherently  inefficient 
and  breeds  cynicism  among  participants. 

Each  of  these  systems  may  have  had  an  un¬ 
intended  consequence  of  increasing  the  de¬ 
mand  for  analysis,  the  capability  to  do  it,  and 
its  use  in  devising  solutions  to  public  policy 
problems.  Indeed,  this  may  be  viewed  as  the 
most  lasting  legacy  of  the  "rational  budgeting" 
movement.  Agency  planning  offices  suddenly 
found  that  their  products  were  in  demand.  A 
new  demand  also  developed  for  personnel  with 
policy  analysis  skills.  As  a  result,  even  after 
these  rational  budgeting  systems  had  faded 
from  view,  their  impact  could  be  seen  in  the 
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staff  that  were  hired  to  do  policy  analysis.  Not 
only  have  these  past  efforts  increased  the  abil¬ 
ity  of  the  bureaucracy  to  evaluate  program 
success,  they  may  make  it  easier  to  put  future 
systems  into  effect. 

Mandating  effectiveness  budgeting  has  pit- 
falls  that  must  be  explicitly  recognized  in  de¬ 
signing  any  system  that  is  intended  to  link 
measurement  and  budgeting.  Experience 


yields  three  important  lessons.  First,  commit¬ 
ment  to  the  measurement  system  by  agency 
personnel  at  all  levels  is  necessary  if  it  is  to  be 
successful.  Second,  the  data  that  are  collected 
must  be  used.  Third,  the  design  of  budget  sys¬ 
tems  must  take  into  account  the  difficulties  in 
linking  the  measurement  of  outcomes  to  bud¬ 
geting.  Time  must  be  devoted  to  clarifying 
these  linkages. 


Chapter  Four 


The  Experience  of  Federal  Agencies 
with  Performance  Measurement 


Many  federal  agencies  already  use 
measurements  to  assist  with  their 
day-to-day  management,  their  bud¬ 
get  justifications,  and  their  reporting  to  exter¬ 
nal  constituencies.  Important  issues  for  the 
current  debate  are  the  extent  to  which  mea¬ 
surements  are  already  used  in  federal  agen¬ 
cies  and  the  insights  that  current  practice  can 
provide  about  the  potential  for  using  perfor¬ 
mance  measurement  for  budgeting.  In  exam¬ 
ining  these  issues,  two  questions  arise  that 
will  be  addressed  by  reviewing  the  results  of  a 
General  Accounting  Office  study  of  federal 
practices  and  case  studies  of  individual  agen¬ 
cies.  First,  to  what  extent  are  federal  agen¬ 
cies  focusing  attention  on  the  results  of  pro¬ 
grams  rather  than  only  on  activities?  Second, 
to  what  extent  are  performance  measures 
used  for  budgeting?  Knowing  about  the  state 
of  performance  measures  within  these  agen¬ 
cies  should  reveal  the  implications  of  using 
measurement  throughout  the  agencies  of  the 
federal  government. 


GAO  Study 
of  Performance 
Measurement 

In  response  to  a  request  from  the  Chairman 
(Senator  John  Glenn)  and  Ranking  Minority 
Member  (Senator  William  Roth)  of  the  Senate 
Committee  on  Governmental  Affairs,  GAO 
surveyed  federal  agencies  to  evaluate  the 
state  of  performance  measurement  in  the  fed¬ 


eral  government.  Surveys  were  completed  by 
102  agencies,  each  of  which  had  more  than 
1,000  employees  or  over  $500  million  in  an¬ 
nual  outlays.  These  agencies  comprised  87 
percent  of  all  federal  employees  and  92  per¬ 
cent  of  total  outlays  in  fiscal  year  1990. 

GAO  asked  agencies  to  report  on  what  per¬ 
formance  measures  (if  any)  they  use  as  a  part 
of  their  operations  and  how  they  use  them  in 
management.  The  survey  concentrated  on  the 
types  of  measurements  used,  their  users,  the 
impetus  for  their  development,  and  top  man¬ 
agers'  use  of  and  satisfaction  with  the  mea¬ 
surements.1 

Agencies  reported  to  GAO  that  they  are  us¬ 
ing  a  wide  array  of  performance  measures.  Al¬ 
though  a  majority  of  the  agencies  said  they 
had  a  strategic  plan  that  sets  out  goals  or  ob¬ 
jectives,  only  about  50  percent  use  their  mea¬ 
sures  to  gauge  their  progress  in  achieving 
those  goals.  The  majority  of  agencies  reported 
that  they  collect  results-oriented  (outcome) 
measurements. 

Seventy  percent  of  agency  managers  who 
responded  said  that  performance  measures 
help  them  make  budget  decisions,  manage 
programs,  assure  accountability,  and  measure 
program  results  or  outcomes.  There  was  also 
general  agreement  (over  80  percent)  among 
respondents  that  their  organization's  perfor- 


1.  These  results  are  reported  in  General  Accounting  Office, 
Program  Performance  Measures:  Federal  Agency  Collec¬ 
tion  and  Use  of  Performance  Data,  GAO/GGD-92-65 
(May  4, 1992). 
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mance  measuring  methods  are  timely,  reli¬ 
able,  or  detailed  in  content. 

Agency  reports  to  GAO,  however,  may  have 
overstated  the  use  of  performance  data  by  fed¬ 
eral  agencies.  There  was,  for  example,  some 
inconsistency  in  agency  responses  concerning 
the  use  of  outcome  measures,  which  raises 
questions  about  whether  agencies  understood 
the  survey  queries  or  whether  their  responses 
truly  reflect  their  use  of  performance  mea¬ 
sures. 

In  fact,  GAO  questioned  the  validity  of  the 
survey  results  and  attempted  to  verify  re¬ 
sponses  by  visiting  several  agencies  and  try¬ 
ing  to  determine  just  how  much  these  agencies 
were  using  performance  measures.  The  re¬ 
sults  of  this  follow-up  by  GAO  confirmed  in¬ 
vestigators'  impression  that  the  survey  data 
generally  overstated  the  use  of  performance 
measures  by  federal  agencies.  That  is  prob¬ 
ably  the  result  of  the  limitations  of  survey  re¬ 
search  to  adequately  assess  practices  in  com¬ 
plex,  hard-to-define  areas. 


CBO’s  Detailed  Case 
Studies 

The  Congressional  Budget  Office  conducted 
several  detailed  studies  of  agencies  and  their 
use  of  performance  measurement.  The  agen¬ 
cies  were  selected  on  the  basis  of  discussions 
with  staff  overseeing  the  GAO  survey,  and  on 
other  reports  concerning  which  federal  agen¬ 
cies  were  already  using  performance  measure¬ 
ment.  Furthermore,  CBO  attempted  to  choose 
a  range  of  agency  types  that  would  represent 
federal  agencies  as  a  whole.  The  six  agencies 
evaluated  were  the  Environmental  Protection 
Agency,  the  Internal  Revenue  Service,  the  De¬ 
partment  of  Labor's  Employment  and  Train¬ 
ing  Administration,  the  Farmers  Home  Ad¬ 
ministration,  the  Department  of  Defense 
(business  operations),  and  the  Public  Health 
Service's  Healthy  People  2000  program. 


The  case  studies  addressed  three  general  is¬ 
sues: 

o  The  extent  to  which  agencies  have  devel¬ 
oped  clear  objectives  or  goals  for  their  pro¬ 
grams,  since  performance  measures  have 
limited  usefulness  without  such  plans  in 
place; 

o  The  types  of  measures  that  the  agency  is 
using  and  how  they  are  calculated,  and 
how  much  the  measures  relate  to  the  agen¬ 
cy's  mission  or  goals;  and 

o  Whether  these  agencies  are  using  perfor¬ 
mance  measurement  data  as  a  tool  to  fa¬ 
cilitate  their  budget  decisionmaking  pro¬ 
cesses. 

Environmental  Protection 
Agency 

The  Environmental  Protection  Agency  (EPA) 
has  used  various  measures  of  performance  for 
the  past  decade.  The  agency  also  uses  an  an¬ 
nual  planning  process  that  outlines  goals  in 
the  measured  areas.  In  the  past,  the  measures 
have  focused  on  activities  (for  example,  hand¬ 
ing  out  permits)  rather  than  outcomes  (im¬ 
proving  the  environment). 

The  goal  of  the  EPA  planning  process  is  to 
reshape  priorities  so  that  the  agency  can  con¬ 
centrate  on  problems  that  pose  the  greatest 
environmental  risk.  The  agency  defines  per¬ 
formance  targets  and  goals  covering  several 
years  and  is  developing  measurements  that 
can  adequately  gauge  agency  progress. 

This  effort  has  tried  to  change  the  focus  of 
performance  measurement  from  one  of  regula¬ 
tory  inputs  to  one  of  environmental  changes. 
Some  of  the  previous  emphasis  on  measuring 
regulatory  activity  arose  from  EPA's  need  to 
conform  to  statutory  requirements.  By  focus¬ 
ing  on  environmental  outcomes  instead  of  reg¬ 
ulatory  activities,  EPA  employees  may  now 
use  statutes  as  a  tool  to  achieve  desired  envi- 
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ronmental  outcomes  rather  than  view  adher¬ 
ence  to  the  law  as  an  end  unto  itself. 

According  to  agency  staff,  four  kinds  of  in¬ 
dicators  are  best  suited  to  measure  EPA  per¬ 
formance; 

o  Activities  undertaken  by  the  programs 
themselves; 

o  Releases  of  pollutants  into  the  environ¬ 
ment; 

o  Ambient  pollution  levels;2  and, 

o  Information  about  environmental  effects.3 

Although  the  first  type  clearly  measures  out¬ 
put,  the  other  three  measures  are  part  of  a 
conscious  effort  by  EPA  to  expand  its  use  of 
outcome  measures.  EPA  believes,  however, 
that  output  measures  can  still  play  an  impor¬ 
tant  part  in  gauging  agency  progress  toward 
measurable  environmental  goals.  Further, 
EPA  staff  noted  that  this  planning  process  has 
made  agency  employees  more  willing  to  have 
their  performance  measured  against  envi¬ 
ronmental  outcomes.  They  noted  that  the  rel¬ 
atively  long  tenure  of  past  EPA  administrator 
William  Reilly  made  using  the  outcome- 
oriented  approach  much  easier. 

One  problem  that  plagues  EPA  in  the  de¬ 
sign  of  its  outcome  measures  is  the  question  of 
causality.  In  many  instances,  outcome  mea¬ 
sures  are  difficult  to  design  because  it  is  hard 
to  ascertain  whether  EPA  "causes"  marginal 
changes  in  the  environment.  For  example,  it 
is  difficult  to  take  into  account  all  of  the  vari¬ 
ables  that  contribute  to  air  quality. 

EPA  budgeting  has  been  tied  very  closely  to 
its  system  of  output  measurements.  Agency 
managers  have  been  able  to  tie  a  given  level  of 
regulation  to  the  work  levels  needed  to  carry  it 


2.  This  indicator  records  conditions  in  the  environment 
caused  by  the  release  of  pollutants  that  may  contribute 
to  adverse  environmental  effects. 

3.  This  indicator  records  effects  on  human  health  and  eco¬ 
logical  damage. 


out.  EPA  has  not  used  outcome  measures  for 
budgeting,  which  would  imply  a  very  different 
focus.  EPA  hopes  to  target  its  resources  to¬ 
ward  areas  that  are  assessed  as  "highest  risk" 
and  divert  precious  money  away  from  rela¬ 
tively  high-priced  but  "low-risk"  problems 
such  as  Superfund  cleanups.  EPA  plans  to  de¬ 
sign  much  of  its  1994  budget  around  these  risk 
reduction  goals  instead  of  the  old  output-based 
funding  targets. 

Internal  Revenue  Service 

Although  the  Internal  Revenue  Service  (IRS) 
has  used  various  performance  measures  since 
the  1960s,  it  started  a  strategic  planning  pro¬ 
cess  in  earnest  in  1986.  Three  major  objec¬ 
tives  have  been  defined  as  keys  to  the  success 
of  the  IRS: 

o  Increasing  voluntary  taxpayer  compli¬ 
ance; 

o  Reducing  taxpayer  burden;  and 

o  Improving  productivity  and  customer  sat¬ 
isfaction. 

During  its  strategic  planning  process,  the 
IRS  discovered  that  most  of  its  measurements 
are  of  outputs,  such  as  the  number  of  audits 
carried  out  or  the  number  of  returns  processed 
in  a  year.  These  measures  enabled  the  IRS  to 
compare  its  performance  in  these  areas  with 
that  in  previous  years,  and  it  often  used  them 
to  set  targets  for  future  years.  But  these  data 
were  not  adequate  for  informing  the  IRS  about 
how  its  actions  were  affecting  taxpayers. 

The  IRS  has  developed  measures  of  results 
on  only  a  limited  basis.  For  example,  it  devel¬ 
oped  a  system  of  measuring  rates  of  accuracy 
for  toll-free  telephone  tax  assistance,  refunds, 
adjustments,  correspondence,  and  the  process¬ 
ing  of  returns.  In  addition,  the  IRS  hopes  to 
design  methods  of  measuring  citizen  satisfac¬ 
tion  (via  surveys)  and  voluntary  compliance. 
Although  the  current  set  of  measurements 
emphasize  process  and  taxpayer  service,  a  few 
focus  on  the  results  of  IRS  enforcement  activ- 
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ity.  In  addition  to  measuring  the  IRS's  ability 
to  encourage  taxpayer  compliance  through  as¬ 
sistance  and  education,  measurements  of  en¬ 
forcement  would  help  in  finding  out  whether 
the  IRS’s  actions  are  actually  increasing  it. 

Although  performance  indicators  are  re¬ 
ported  in  the  President's  budget  and  to  the 
Congress  with  budget  justifications,  the  IRS 
has  not  used  them  directly  to  influence  the  al¬ 
location  of  resources.  The  IRS  is  moving  to  de¬ 
velop  more  and  better  measures  of  perfor¬ 
mance  for  management. 

Employment  and  Training 
Administration 

Through  its  Employment  and  Training  Ad¬ 
ministration,  the  Department  of  Labor  man¬ 
ages  the  Job  Training  Partnership  Act 
(JTPA).  The  purpose  of  this  1982  act  is  to  give 
disadvantaged  youths  and  adults  important 
job  skills  and  help  place  them  in  jobs.4 

To  assist  in  carrying  out  JTPA's  purpose, 
the  act  calls  for  mandatory  performance  stan¬ 
dards,  monetary  incentives  to  reward  local 
programs  that  exceed  their  standards,  and 
sanctions  for  those  that  fail  to  meet  their  stan¬ 
dards.  Thus,  in  addition  to  its  basic  allocation, 
a  local  program  can  receive  additional  funds  to 
expand  or  improve  its  program. 

Performance  standards  for  programs  serv¬ 
ing  adults  measure  the  employment  status 
and  earnings  of  those  who  complete  the  pro¬ 
gram.  These  data  are  collected  approximately 
three  months  after  individuals  leave  the  pro¬ 
gram.  Previously,  JTPA  measured  success 
based  on  the  ability  of  programs  to  place  par¬ 
ticipants  immediately  after  they  complete 
their  programs.  But  that  created  counterpro¬ 
ductive  incentives.  Many  programs  concen¬ 
trated  on  enrolling  participants  who  were  the 
most  prepared  for  employment,  ignoring  those 
who  were  more  disadvantaged  and  in  need  of 
training.  This  practice  is  commonly  referred 


4.  P.L.  97-300,  Sec.  2.  (Job  Training  Partnership  Act). 


to  as  "creaming."  JTPA  revised  its  perfor¬ 
mance  measures  in  1990  to  emphasize  job  re¬ 
tention,  as  measured  by  the  proportion  of 
those  who  complete  the  program  and  are  em¬ 
ployed  13  weeks  later.  This  measure  provides 
for  a  longer-term  assessment  of  the  program's 
effect.  The  creaming  problem  remains,  al¬ 
though  amendments  to  JTPA  have  attempted 
to  address  it  by  targeting  funds  toward  var¬ 
ious  disadvantaged  groups. 

The  annual  planning  process  at  JTPA  is  de¬ 
centralized.  Each  state  combines  the  plans  of 
its  individual  local  or  service  delivery  areas, 
or  SDAs,  into  a  single  statewide  plan,  which  is 
sent  to  the  national  office  for  approval.  (SDAs 
are  the  geographic  subunits  responsible  for 
carrying  out  JTPA  activities.)  The  national 
office  evaluates  these  plans  on  the  basis  of 
their  compliance  with  statutes,  regulations, 
and  consistency  with  the  program's  overall 
goals;  it  does  not  use  them  to  monitor  perfor¬ 
mance  or  ability  to  meet  planning  targets. 

To  encourage  SDAs  and  states  to  perform  at 
or  in  excess  of  their  yearly  performance  tar¬ 
gets,  JTPA  has  a  built-in  system  of  monetary 
incentives.  Each  state  must  set  aside  6  per¬ 
cent  of  its  Title  II-A  allocation  for  rewarding 
local  programs  and  providing  technical  assis¬ 
tance  to  those  that  need  to  improve  their  per¬ 
formance.  Each  governor  controls  the  formula 
for  distributing  the  6  percent  set-aside  among 
SDAs  within  a  state.  Within  certain  limits, 
governors  may  give  "bonuses"  to  programs 
that  perform  well.  Governors  are  authorized 
to  impose  sanctions,  such  as  reorganization,  if 
an  SDA  falls  below  performance  standards  for 
two  years.  Most  governors  would  rather  give 
SDAs  technical  assistance  than  impose  sanc¬ 
tions.  Performance  measures  are  not  linked  to 
JTPA's  resource  allocation  process.  The  vast 
majority  of  SDA  funding  is  based  on  need. 

Furthermore,  JTPA  managers  do  not  be¬ 
lieve  that  the  resource  allocation  process 
should  be  linked  to  performance  standards. 
They  believe  that  performance  measures  are 
more  of  a  tool  for  agency  management  and  less 
of  a  tool  for  budgeting.  Their  experience  sug¬ 
gests  that  a  performance-based  budgeting  ap- 
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proach  can  produce  unintended  effects.  More¬ 
over,  comparisons  among  human  resource  pro¬ 
grams  that  are  based  solely  on  performance 
outcomes  do  not  necessarily  identify  the  most 
effective  programs.  Program  evaluations,  con¬ 
versely,  can  track  longer-term  program  effects 
for  a  given  cohort  of  participants.  These  eval¬ 
uations  represent  more  in-depth  attempts  by 
researchers  to  determine  the  true  effects  of  a 
program,  often  by  tracking  the  experiences  of 
a  small  group  of  participants.  Thus,  they  can 
more  easily  demonstrate  the  marginal  effect 
that  a  given  program  may  have  on  clients. 
The  data  from  such  studies,  however,  are  often 
not  timely;  program  design  and  operation  of¬ 
ten  change  before  results  become  available. 
For  this  reason,  the  results  of  evaluations  are 
also  limited  in  their  ability  to  influence  re¬ 
source  allocation. 


Farmers  Home  Administration 

The  Farmers  Home  Administration  (FmHA), 
founded  in  1935,  originally  assisted  family 
farmers  in  regaining  self-sufficiency  after  the 
Depression.  More  recently,  the  Congress  has 
extended  the  responsibility  of  FmHA  to  cover 
nonfarm  programs  in  rural  areas  as  well. 
FmHA  makes  direct  loans,  guarantees  loans 
made  by  private  lenders,  and  makes  a  limited 
number  of  grants.5 

In  its  June  1992  FmHA  Strategic  Business 
Plan,  FmHA  set  out  a  list  of  goals  establishing 
agency  priorities  for  the  next  five  years.  The 
report  reflects  FmHA's  most  important  goals 
in  the  short  run-improving  its  credit  quality, 
better  managing  its  loan  portfolio,  and  mak¬ 
ing  sure  that  it  serves  as  the  lender  of  last  re¬ 
sort  for  farmers. 

On  the  basis  of  these  national  goals,  FmHA 
sets  performance  targets  for  each  of  its  50 
state  offices.  If  the  state  offices  meet  their 


goals,  FmHA  will  have  achieved  its  national 
performance  targets.  FmHA's  state  and  na¬ 
tional  offices  negotiate  the  targets  for  state 
performance.  The  national  office  notifies  the 
states  when  they  must  take  corrective  action 
to  achieve  performance  target  levels  and  gives 
state  offices  flexibility  to  design  appropriate 
strategies  to  achieve  these  targets. 

For  the  most  part,  FmHA  uses  output  mea¬ 
sures.  These  measures  include  the  number  of 
annual  performance  reports  received  by  state 
FmHA  offices,  the  number  of  loan  guarantees 
as  a  percentage  of  total  obligations,  or  the 
number  of  properties  sold.  One  measure  that 
could  be  called  an  outcome  measure,  given  the 
stated  goals  of  FmHA,  is  the  number  of  bor¬ 
rowers  who  change  from  FmHA  lending  to 
other  forms  of  credit.  FmHA  calls  that  credit 
"graduation."  In  other  words,  borrowers  have 
improved  their  credit  enough  to  be  able  to  bor¬ 
row  from  private  sources  instead  of  having  to 
use  FmHA  (the  "lender  of  last  resort"). 

FmHA  has  developed  a  scoring  system  to 
evaluate  the  performance  of  each  state.  The 
national  office  links  the  state's  performance 
rankings  directly  with  its  bonus  system  for  up¬ 
per  management.  The  better  a  state  does  in 
achieving  all  of  the  FmHA  goals,  the  larger 
the  size  of  its  bonus  pool. 

Even  though  the  bulk  of  FmHA  funding  is 
awarded  to  states  on  the  basis  of  legal  formu¬ 
las  (the  variables  that  contribute  to  the  calcu¬ 
lation  of  appropriate  funding  levels  include 
the  economic  and  demographic  characteristics 
of  a  state's  rural  areas),  FmHA  believes  that 
the  achievement  of  state  objectives  will  save 
the  government  money  by  reducing  loan 
losses.  In  other  words,  by  pushing  states  to  re¬ 
duce  delinquencies  and  encourage  borrowers 
to  "graduate"  to  private  lenders,  FmHA  is  try¬ 
ing  to  reduce  the  costs  of  its  programs  as  well 
as  the  loss  that  failed  loans  impose  on  the  U.S. 
Treasury. 


5.  President  Clinton's  budget  for  fiscal  year  1994  recom¬ 
mends  that  the  name  of  the  FmHA  be  changed  to  the 
Farm  Service  Agency.  This  new  agency  would  combine 
the  FmHA's  current  functions  with  various  other  activi¬ 
ties  not  now  under  its  jurisdiction. 
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Department  of  Defense- 
Business  Operations 

The  Department  of  Defense  (DoD)  is  the  larg¬ 
est  federal  agency,  employing  more  people 
than  any  other  department.  The  department 
uses  various  surrogate  measures  to  assess 
progress  toward  its  outcomes.  For  example, 
while  the  ultimate  outcomes  of  defense  are 
concerned  with  national  security,  many  of  the 
measurements  that  the  department  uses 
gauge  military  readiness.  Many  of  these  mea¬ 
surements  are  classified,  and  there  is  no  ex¬ 
plicit  linkage  between  these  macro-level  mea¬ 
sures  and  the  budget. 

The  department  has,  however,  been  defin¬ 
ing  measures  of  both  efficiency  and  effective¬ 
ness  for  its  internal  service,  or  business,  ac¬ 
tivities.  This  process  is  tied  to  the  develop¬ 
ment  of  a  unit  cost  structure  for  DoD  and  is  al¬ 
so  associated  with  the  development  of  the  De¬ 
fense  Business  Operations  Fund  (DBOF)  and 
the  application  of  the  Chief  Financial  Officers 
Act. 

The  DBOF  was  established  on  October  1, 
1991.  The  purpose  of  the  fund  is  to  consolidate 
all  revolving  funds  within  DoD,  with  the  goal 
of  giving  department  managers  the  manage¬ 
ment  tools  and  information  necessary  to  pro¬ 
vide  services  at  the  lowest  cost.  Clients  of  the 
fund  (the  military  services)  are  to  be  charged 
rates  that  recover  the  full  cost  of  providing  a 
given  level  of  service.  Given  that  DoD's  bud¬ 
get  is  being  reduced,  this  effort  is  particularly 
timely.  For  fiscal  year  1992,  the  fund  had  $69 
billion  in  collections  and  $72  billion  in  dis¬ 
bursements. 

DoD  has  recently  published  guidelines  for 
the  development  of  performance  measures  for 
the  DBOF.  These  measures  will  focus  on  both 
efficiency  and  effectiveness.  The  department 
is  planning  to  use  fairly  standard  measures  of 
efficiency  (cost  per  unit  of  output)  for  the 
DBOF.  Effectiveness  measures  will  focus  on 
timeliness  (how  quickly  the  item  is  delivered), 
accuracy  (assuring  that  the  right  item  is  deliv¬ 
ered),  and  customer  satisfaction.  The  specific 
measures  are  still  under  development. 


The  measures  are  being  developed  in  re¬ 
sponse  to  legislated  requirements.  Not  only 
does  the  CFO  act  require  chief  financial  offi¬ 
cers  of  agencies  to  develop  performance  mea¬ 
sures,  but  the  fiscal  year  1993  Defense  Au¬ 
thorization  Act  specifically  required  perfor¬ 
mance  measures  for  the  Defense  Business  Op¬ 
erations  Fund  by  March  1, 1993. 

The  key  objective  of  the  DoD  "performance 
budgeting"  initiative  is  to  create  incentives  for 
managers  to  reduce  costs  but  still  achieve 
operational  performance  goals.  The  initiative 
is  also  aimed  at  focusing  strategies  for  improv¬ 
ing  the  processes  for  considering  capital  in¬ 
vestments  and  other  policies,  and  for  improv¬ 
ing  efficiency  and  effectiveness.  In  practice, 
this  means  that  targets  will  be  set  for  oper¬ 
ational  performance,  and  reducing  costs  is  a 
desired  outcome  only  insofar  as  service  does 
not  suffer.  As  the  DoD  effort  is  still  embry¬ 
onic,  it  is  too  early  to  tell  whether  the  agency 
will  be  able  to  achieve  this  accommodation  of 
efficiency  and  effectiveness. 


The  Public  Health  Service’s 
Healthy  People  2000  Program 

In  1976,  Title  XVII  of  the  Public  Health  Ser¬ 
vice  Act  (PHS)  directed  the  Department  of 
Health  and  Human  Services  (HHS)  to  estab¬ 
lish  goals  for  the  prevention  of  disease  and  the 
promotion  of  strategies  to  achieve  those  goals. 
The  Office  of  Disease  Prevention  and  Health 
Promotion,  also  created  under  the  act,  was 
given  the  task  of  coordinating  prevention  poli¬ 
cies  and  programs  within  HHS  that  help 
achieve  these  goals.  Healthy  People:  The  Sur¬ 
geon  General's  Report  on  Health  Promotion 
and  Disease  Prevention ,  published  in  1979,  es¬ 
tablished  broad  national  goals  for  reducing 
mortality  rates  in  four  age  groups  from  birth 
to  age  65.  By  1990,  which  was  the  target  year 
for  the  achievement  of  these  goals,  three  out  of 
the  four  had  been  achieved.  Thirty-two  per¬ 
cent  of  those  national  objectives  were  achieved 
by  1990,  and  another  34  percent  of  the  objec¬ 
tives  showed  progress. 
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The  Public  Health  Service  repeated  this 
process  in  1987,  designing  a  program  called 
Healthy  People  2000,  which  established  an  ex¬ 
tensive  set  of  national  health  objectives  to  be 
achieved  by  the  turn  of  the  century.  Healthy 
People  2000  has  three  broad  goals: 

o  To  increase  the  span  of  healthy  life  for  peo¬ 
ple  in  the  United  States; 

o  To  reduce  (and  finally  eliminate)  health 
disparities  among  subpopulations  of  the 
United  States;  and, 

o  To  give  all  people  in  the  United  States  ac¬ 
cess  to  preventive  services. 

Healthy  People  2000  work  groups  developed 
300  measurable  objectives  that  can  be  sepa¬ 
rated  into  22  priority  areas.  These  measures 
can  be  categorized  into  three  groups: 

o  Health  outcome  measures,  such  as  the 
reduction  of  deaths  caused  by  heart  dis¬ 
ease.  The  quality  of  such  data  is  relatively 
accurate,  but  the  direct  effect  of  PHS  ac¬ 
tions  on  such  statistics  is  hard  to  assess. 

o  Risk  reduction  measures,  such  as  im¬ 
munization  rates  or  the  number  of  people 
who  quit  smoking.  The  data  for  this  group 
of  measurements  is  less  exact  than  that  for 
outcome  measurements  because  much  of 
the  information  is  from  self-reported 
health  surveys. 

o  Service  and  health  protection  mea¬ 
sures,  such  as  the  percentage  of  women 
who  receive  prenatal  care  in  the  first  tri¬ 
mester  of  pregnancy. 

Although  the  data  on  the  quality  of  service 
and  health  protection  is  not  as  good  as  in  the 
other  two  measurement  groups,  PHS  actions 
in  these  areas  are  most  likely  to  have  the 
greatest  direct  effect.  This  points  up  a  recur¬ 
ring  paradox  for  PHS:  areas  in  which  its  ac¬ 
tions  can  have  the  greatest  effect  on  health 
may  be  difficult  to  measure,  and  areas  that 
may  be  easiest  to  measure  are  those  on  which 


PHS's  actions  have  the  least  demonstrable 
marginal  impact. 

It  is  difficult  for  PHS  to  link  performance 
and  overall  health  results  directly,  and  thus  it 
is  doubly  difficult  to  bestow  rewards  or  pun¬ 
ishments  on  the  basis  of  performance  mea¬ 
sures.  The  task  is  further  complicated  by  the 
fact  that  the  national  results  in  these  health 
areas  are  affected  by  the  actions  of  the  private 
sector  as  well  as  by  those  of  all  three  levels  of 
government. 

The  linkage  between  these  measurements 
and  budgeting  or  resource  allocation  decisions 
is  still  in  the  developmental  stage.  Internally, 
PHS  prepared  its  1994  budget  in  terms  of  sup¬ 
port  for  Healthy  People  2000  objectives.  The 
current  link  between  measurements  and  bud¬ 
gets,  however,  is  not  refined. 

Areas  that  are  easier  to  measure,  however, 
are  better  suited  to  the  linkage  between  mea¬ 
surement  and  resource  allocation.  In  immuni¬ 
zation  programs,  cost-effectiveness  can  be 
analyzed  to  determine  the  amount  of  money 
needed  to  improve  the  immunization  rate  by  a 
certain  amount  and  thus  limit  the  infection 
rate  for  that  disease.  Such  linkages  between 
spending  and  health  effects,  however,  are 
much  more  difficult  for  incurable  diseases  and 
those  for  which  the  causes  are  varied  and  hard 
to  control.  Because  of  such  limitations,  the 
link  between  budget  decisionmaking  pro¬ 
cesses  and  performance  measurement  is  still 
under  development  at  PHS. 


Lessons  Learned 
from  Current 
Practices  of  Agencies 

Several  themes  emerge  from  the  experience  of 
federal  agencies,  including  the  importance  of 
having  a  clearly  defined  mission,  the  difficulty 
of  developing  outcome  measures  for  many  gov¬ 
ernment  functions,  and  the  often  tenuous  ties 
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between  budgeting  and  measures  of  perfor¬ 
mance. 

The  Difficulty  of  Defining  Agency  Missions  or 
Goals  at  the  Federal  Level.  Managers  of  fed¬ 
eral  agencies  can  be  influenced  by  several  au¬ 
thorities,  including  the  White  House  staff,  the 
Office  of  Management  and  Budget,  multiple 
Congressional  committees,  political  appoin¬ 
tees  within  agencies,  and  other  agency  man¬ 
agers.  Because  clearly  defined  goals  and  mis¬ 
sions  are  the  foundation  on  which  perfor¬ 
mance  measurement  is  built,  performance 
measurement  initiatives  are  severely  ham¬ 
pered.  Without  clear  goals,  it  is  difficult  for  an 
agency  to  measure  how  well  it  is  performing 
its  task.  Before  one  can  talk  about  perfor¬ 
mance  measurement,  agreement  must  be 
reached  on  agency  goals  and  priorities. 

The  Difficulty  of  Linking  Performance  Mea¬ 
sures  and  the  Budget  Process.  None  of  the 
agencies  studied  used  performance  measure¬ 
ment  to  make  decisions  about  the  level  of  re¬ 
sources  that  the  program  received  in  the  bud¬ 
get  process,  although  some  intended  to  do  so. 
Performance  measurement  is  used  in  some 
cases  to  determine  the  allocation  of  monetary 
awards  to  employees  or  subunits  (for  example, 
in  JTPA  and  FmHA),  but  such  reallocations 
are  at  the  margins.  Basic  aspects  of  agencies' 
budgets  are  not  determined  by  the  relation¬ 
ship  between  inputs  and  outcomes. 

Measurements  of  Outcomes  Are  Few  and  Far 
Between.  Even  in  agencies  that  are  regarded 
as  the  best  examples  of  federal  performance 
measurement,  outcome  measures  are  rare. 
JTPA  may  come  the  closest  to  the  mark  with 
its  measurement  of  job  retention,  but  even  it  is 
relatively  short-term  (measured  13  weeks 
after  the  training  has  ended)  and  does  not  give 
a  sense  of  the  marginal  impact  that  the  pro¬ 
gram  has  on  its  trainees  in  the  long  run.  Ac¬ 
cording  to  JTPA  staff,  a  true  analysis  of  the 
program's  effect  would  necessitate  a  program 
evaluation  that  addressed  this  factor.  In  fact, 
program  evaluations  may  even  contradict  the 
"results"  based  on  performance  measures. 
Moreover,  even  program  evaluations  gener¬ 
ally  cannot  separate  out  alternative  explana¬ 


tions  for  program  outcomes.6  Many  agencies 
(such  as  EPA)  find  it  difficult  to  make  a  direct 
connection  between  what  they  do  and  what 
happens  as  a  result. 

Function  Matters.  Certain  governmental 
tasks  or  functional  areas  are  simply  less  ame¬ 
nable  to  effective  performance  measurement. 
Programs  that  provide  direct  services  that  are 
under  the  control  of  an  agency  are  easier  to 
measure  than  others,  such  as  regulatory  and 
grant  activities.  Within  the  IRS,  for  example, 
certain  tasks,  such  as  the  accuracy  of  process¬ 
ing  tax  returns,  are  easy  to  quantify  and  mea¬ 
sure;  others,  such  as  the  accuracy  of  taxpayer 
assistance,  are  more  difficult  to  quantify  or 
measure  in  a  meaningful  way.  EPA  and  PHS 
have  trouble  discerning  the  marginal  impacts 
of  many  of  their  programs.  In  fact,  it  is  diffi¬ 
cult  to  determine  what  constitutes  "good"  or 
"bad"  performance  by  agency  subunits  in 
these  hard-to-measure  areas,  ha  general,  per¬ 
formance  systems  apparently  work  best  where 
there  is  direct  accountability  and  where  cause- 
and-effect  relationships  can  be  clearly  drawn 
between  what  the  agency  does  (outputs)  and 
what  it  wants  to  happen  (outcomes).  That 
does  not  mean  that  bad  performance  should  be 
expected  in  hard-to-measure  areas,  but  rather 
that  the  indicators  of  success  may  be  more  in¬ 
tangible  in  some  cases  than  in  others. 

Commitment  of  Top  Managers  Directly  Affects 
the  Success  of  Performance  Measurement  Sys¬ 
tems.  The  cases  of  FmHA  and  EPA  illustrate 
the  general  truism  that  the  success  of  perfor¬ 
mance  measurement  programs  (as  in  any  oth¬ 
ers)  depends  directly  on  upper  management's 
commitment  to  them.  In  the  case  of  EPA,  per¬ 
formance  measurement  initiatives  in  the  risk 
reduction  area  have  been  aided  considerably, 
according  to  EPA  managers,  by  the  presence 
of  an  EPA  administrator  who  has  held  office 
for  a  relatively  long  time.  At  FmHA,  the  pres¬ 
ence  of  a  committed,  hands-on  management 
team  has  made  the  agencywide  application  of 
its  performance  measurement  system  more 


6.  For  an  expanded  discussion,  see  Robert  Behn,  Leader¬ 
ship  Counts  (Cambridge:  Harvard  University  Press, 
1991). 
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complete  and  effective.  If  upper  management 
is  not  committed  to  performance  measure¬ 
ment,  or  is  transitory,  the  chances  that  perfor¬ 
mance  measurement  might  gain  a  foothold  in 
federal  agencies  are  greatly  reduced. 


Conclusion 

Performance  measurement  within  federal 
agencies  may  not  be  as  prevalent  as  agen¬ 


cies’  responses  to  the  GAO  survey  would  indi¬ 
cate.  Some  impressive  strides,  however,  have 
been  made  in  the  agencies  visited  by  CBO. 
Even  within  those  agencies  that  are  good  ex¬ 
amples  of  performance  measurement,  it  has 
been  difficult  to  develop  outcome  measures 
and  to  connect  performance  measurement  and 
budgeting.  Any  performance  measurement 
system-whether  it  is  governmentwide  or  de¬ 
signed  on  an  agency-by-agency  basis-should 
incorporate  the  lessons  learned  by  these  agen¬ 
cies  and  others  that  have  put  performance 
measurement  systems  into  effect. 


Chapter  Five 

Evaluating  Proposals  for  Measuring 
and  Budgeting  for  Federal  Performance 


The  measurement  of  performance  by  Given  a  mix  of  legislation  and  executive 

federal  agencies  and  the  conversion  of  branch  commitment,  there  are  still  two  main 

the  federal  government  to  a  system  of  obstacles  to  improving  performance  measure- 

performance-based  budgeting  have  recently  ment  and  tying  those  measures  to  the  budget 

drawn  serious  scrutiny.  Principal  among  the  process.  First,  it  is  difficult  to  develop  mea- 

reasons  for  the  attention  are  three  separate  sures  that  accurately  reflect  the  performance 

efforts:  the  reforms  of  federal  financial  man-  of  federal  programs,  especially  since  nonfed- 

agement;  the  passage  of  legislation  in  the  eral  actors  largely  determine  the  success  of  so 

103rd  Congress  covering  performance  mea-  many  of  these  programs.  Second,  changing 

surement;  and  a  review  of  the  operation  of  the  the  way  that  policies  are  made  in  order  to 

executive  branch.  make  them  more  responsive  to  the  measure¬ 

ment  of  outcomes  would  require  a  stronger  de- 
Few  people  would  disagree  that  an  in-  mand  for  them  by  policymakers  than  cur- 

creased  focus  on  the  results  of  federal  pro-  rently  exists, 

grams  might  produce  benefits.  Translating 

these  hoped-for  benefits  into  reality,  however,  The  payoffs  that  may  come  about  from 

is  difficult.  This  study  concludes  that  obsta-  performance-based  budgeting  are  limited.  Us- 

cles  remain  to  improving  the  measurement  of  ing  performance  measures  for  allocating  gov- 

federal  performance  and  adopting  perfor-  emment  resources  requires  a  transformation 

mance-based  budgeting.  Even  if  these  obsta-  of  current  processes,  which  are  heavily  focused 

cles  can  be  overcome,  however,  no  perfor-  on  inputs,  into  an  outcome-oriented  system.  It 

mance  measurement  effort  will  succeed  with-  is  impossible  to  effectuate  such  a  system  by 

out  a  combination  of  legislative  and  executive  simply  demanding  it;  attempts  to  create  one 

efforts  to  improve  financial  and  program  man-  by  fiat  have  failed  in  the  past.  Although  it 

agement.  Legislation  is  limited  in  its  ability  may  be  possible  to  foster  such  a  demand  by  im- 

to  bring  about  lasting  change  but  can  assist  in  proving  the  information  available  to  policy- 

some  areas,  such  as  motivating  agencies  to  makers,  it  is  unlikely  that  simply  mandating 

measure  results,  increasing  the  flexibility  of  performance  measurement  will  have  any  real 

program  managers,  and  mandating  that  in-  effect, 

formation  on  program  performance  be  pro¬ 
vided  to  policymakers.  Executive  branch  com-  If  policymakers  demand  information  on  the 

mitment,  however,  is  crucial  in  bringing  about  relationship  between  dollars  and  results,  how- 

any  real  changes  in  agency  management,  ever,  they  could  bring  about  real  changes  in 

through  providing  incentives  for  agency  man-  the  budget  process.  Reporting  information  on 

agers,  improving  information  systems,  and  the  results  of  government  programs  in  govern- 

tailoring  solutions  to  individual  problems.  ment  financial  statements  can  improve  the  ca- 
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pacity  to  create  such  information  as  well  as  de¬ 
velop  a  demand  for  it.  But  the  largest  poten¬ 
tial  for  real  payoffs  may  be  in  the  area  of  agen¬ 
cy  management  of  resources  after  they  are 
budgeted.  This  approach  is  much  more  fruit¬ 
ful  than  performance  measures  for  govern¬ 
mentwide  resource  allocation,  and  it  requires, 
first  and  foremost,  the  commitment  of  political 
and  career  management  to  supporting  the  ef¬ 
forts  of  agency  employees. 


Efforts  Aimed 
at  Improving  Federal 
Performance 
Measurement 

The  renewed  attention  paid  to  performance 
measurement  in  the  federal  government  man¬ 
ifests  itself  in  three  main  (and  related)  ways: 
the  application  of  laws  and  administrative  ef¬ 
forts  designed  to  improve  the  quality  of  federal 
accounting  and  financial  reporting,  the  intro¬ 
duction  of  legislation  that  would  increase  re¬ 
quirements  for  reporting  performance,  and 
President  Clinton’s  recently  announced  per¬ 
formance  review  of  federal  agencies,  which  fo¬ 
cuses  on  improving  the  management  of  the  ex¬ 
ecutive  branch. 

Current  Federal  Efforts 
to  Improve  Accounting 
and  Financial  Reporting 

The  Chief  Financial  Officers  Act  (P.L.  101- 
576)  became  law  in  1990.  It  established  the 
Deputy  Director  for  Management  at  the  Office 
of  Management  and  Budget  as  the  chief  finan¬ 
cial  officer  of  the  federal  government  and  in¬ 
stalled  chief  financial  officers  in  23  of  the  larg¬ 
est  federal  agencies.  The  act's  primary  pur¬ 
pose  was  to  improve  federal  financial  manage¬ 
ment.  But  the  bill  also  included  a  provision 
that  explicitly  addressed  performance  mea¬ 
sures.  This  provision  requires  agency  CFOs  to 
develop  "systematic  measures  of  performance" 


for  programs  in  their  agencies.  It  also  in¬ 
structs  CFOs  "to  prepare  and  submit  to  the 
agency  head  timely  reports"  and  requires  that 
financial  statements  "shall  reflect  results  of 
operations."  In  none  of  these  cases,  however, 
does  the  act  or  the  legislative  history  elabo¬ 
rate  on  how  the  Congress  expects  agencies  to 
respond  to  this  provision. 

Subsequently,  the  Office  of  Management 
and  Budget  (OMB)  instructed  agency  CFOs  to 
prepare  performance  measures  for  fiscal  year 
1992  financial  reports,  which  were  submitted 
to  OMB  on  March  31, 1993.  OMB  initiated  an 
interagency  effort,  designed  to  develop  indica¬ 
tors  that  are  common  to  agencies  as  well  as 
those  that  are  agency-specific.  These  indica¬ 
tors  are  to  be  compared  against  the  agency's 
mission  and  to  concentrate  on  both  outputs 
and  outcomes.  They  are  to  be  used  both  for  in¬ 
ternal  management  and  to  inform  the  public 
and  elected  officials  about  the  accomplish¬ 
ments  of  federal  programs. 

OMB  reviewed  the  preliminary  results  of 
this  effort  in  a  July  1992  memorandum  to 
agency  CFOs.  With  the  exception  of  research 
and  development,  the  teams  for  each  function 
were  able  to  develop  common  measurements. 
Still,  most  other  agencies  had  trouble  develop¬ 
ing  good  measures  of  outcomes.  The  current 
challenge  is  to  expand  the  number  of  agencies 
that  report  such  measurements  and  to  im¬ 
prove  the  quality  of  those  that  they  report  (for 
example,  by  including  more  outcome  mea¬ 
sures). 

A  related  effort  by  the  Federal  Accounting 
Standards  Advisory  Board  is  under  way.  The 
board  is  an  interagency  group  charged  with 
developing  common  accounting  standards  for 
the  federal  government.  It  has  shown  interest 
in  using  performance  measures  for  financial 
reporting  purposes.  In  the  past,  financial  re¬ 
porting  has  been  concerned  primarily  with  the 
consistent  and  accurate  reporting  of  costs.  In 
addition  to  reporting  accurately  how  many 
dollars  are  spent,  it  would  be  useful  to  have  in¬ 
formation  on  what  government  funds  pur¬ 
chase  for  the  public-the  results  of  public  pro¬ 
grams.  This  focus  on  measurement  of  results 
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is  consistent  with  emphasis  that  the  Govern¬ 
mental  Accounting  Standards  Board  places  on 
service  efforts  and  accomplishments  reporting 
at  the  state  and  local  level. 

Legislative  Efforts  to  Expand 
the  Use  of  Performance 
Measures  in  the  Budget 

The  Congress  has  also  turned  its  attention  to 
performance  measurement.  Several  Congres¬ 
sional  committees  have  held  hearings  or  is¬ 
sued  reports  on  performance  measurement  or 
federal  management  in  general,  or  on  prob¬ 
lems  existing  in  particular  agencies.  In  addi¬ 
tion,  many  authorizing  committees  include 
performance  targets  in  reauthorization  bills. 
This  concern  for  measuring  performance  ulti¬ 
mately  led  to  the  introduction  of  several  pieces 
of  legislation  to  mandate  the  development  of 
performance  measures  and  their  use  in  the 
budget  process.  One  of  these  bills,  S.  20, 
passed  the  Senate  on  October  1,  1992;  it  was 
reintroduced  (as  S.  20  in  the  Senate  and  H.R. 
826  in  the  House)  in  the  103rd  Congress.  On 
May  25, 1993,  H.R.  826  passed  the  full  House 
by  unanimous  consent.  S.  20  passed  the  full 
Senate  by  unanimous  consent  on  June  23. 1 * * * S. 
On  July  15,  the  House  passed  S.  20,  clearing 
the  measure  for  the  President’s  signature. 


1.  The  legislative  history  is  as  follows.  On  October  3, 1990, 

Senator  William  Roth  introduced  S.  3154,  the  Federal 

Program  Performance  Standards  and  Goals  Act  of  1990. 

No  action  was  taken  on  this  bill  in  the  101st  Congress, 
and  it  was  reintroduced  aa  S.  20  at  the  beginning  of  the 
102nd  Congress.  Two  separate  hearings  were  held  by 
the  Senate  Committee  on  Governmental  Affairs,  and  a 
committee-reported  substitute  bill  was  passed  by  the 
Senate  on  October  1, 1992.  Several  other  bills  aimed  at 
mandating  performance  measures  were  introduced  in 
the  102nd  Congress  as  well.  During  the  103rd  Congress, 

S.  20  was  reintroduced;  an  almost  identical  bill  (H.R. 
826)  was  introduced  in  the  House  by  John  Conyers, 
Chairman  of  the  Government  Operations  Committee. 
Both  the  Senate  Committee  on  Governmental  Affairs 
and  the  House  Committee  on  Government  Operations 
held  hearings  on  performance  measurement  legislation 
in  the  103rd  Congress,  and  each  committee  reported  bills 
(to  the  full  House  and  full  Senate,  respectively)  in  that 
Congress.  On  May  25,  1993,  H.R.  826  passed  the  full 
House  by  unanimous  consent.  On  June  23,  1993,  S.  20 
passed  the  full  Senate  by  unanimous  consent.  The 
House  approved  S.  20  on  July  15. 


S.  20,  as  introduced,  concentrated  more  on 
the  budget  process  than  does  the  current  ver¬ 
sion.  The  bill  that  ultimately  was  signed  into 
law  in  1993  includes  requirements  for  strate¬ 
gic  planning,  annual  performance  planning 
and  reporting,  and  managerial  flexibility.  It 
also  defines  a  series  of  pilot  projects  as  tests  for 
changes  such  as  performance  budgeting  and 
managerial  flexibility. 

Strategic  Planning.  S.  20  requires  each  agen¬ 
cy  to  develop  a  strategic  plan  for  activities  un¬ 
der  its  jurisdiction.  The  first  of  these  plans 
would  be  submitted  to  the  Office  of  Manage¬ 
ment  and  Budget  by  September  30,  1997. 
Plans  would  cover  at  least  a  five-year  period. 
Strategic  plans  would  include  a  comprehen¬ 
sive  mission  statement,  a  set  of  general  goals 
or  objectives  for  the  program(s),  and  a  list  of 
any  factors  outside  the  agency  that  may  affect 
achievement  of  those  goals  and  objectives. 
Virtually  all  agencies  would  be  covered  by  this 
requirement. 

Annual  Performance  Planning  and  Reporting. 
Beginning  with  fiscal  year  1999,  the  budget 
for  the  U.S.  government  would,  under  this  leg¬ 
islation,  include  a  performance  plan.  These 
plans  would  include  performance  goals  and  in¬ 
dicators  (quantitative,  where  possible)  en¬ 
abling  the  Congress  and  the  public  to  gauge 
whether  agencies  have  complied  with  the 
goals.  Each  agency  would  be  required  to  sub¬ 
mit  a  specific  performance  plan,  on  a  schedule 
to  be  determined  by  OMB,  that  would  cover 
the  major  activities  for  which  it  is  responsible. 

In  addition  to  this  information  reported  in 
the  budget,  beginning  in  fiscal  year  1999  each 
agency  would  be  required  to  submit  program 
performance  reports  to  the  Congress.  The  first 
of  them  would  be  submitted  no  later  than 
March  31,  2000.  These  reports  would  include 
information  comparing  actual  with  planned 
performance,  a  discussion  of  the  success  in 
meeting  goals,  and  remedial  action  if  the  goals 
were  not  met. 

Managerial  Accountability  and  Flexibility 
Waivers.  Agencies  would  be  authorized  under 
the  bill  to  ask  OMB  for  waivers  of  nonstatuto- 


42  USING  PERFORMANCE  MEASURES  IN  THE  FEDERAL  BUDGET  PROCESS 


July  1993 


ry  procedural  requirements.  The  purpose  of 
these  waivers  would  be  to  remove  impedi¬ 
ments  to  agency  managers,  who  would  be  held 
more  accountable  for  results  in  exchange  for 
the  removal  of  administrative  regulations  con¬ 
straining  their  actions.  These  waivers  might 
cover  nonstatutory  personnel  policies  or 
spending  restrictions. 

Pilot  Projects.  The  legislation  would  establish 
several  sets  of  pilot  projects.  The  first  set 
would  include  pilot  tests  for  developing  perfor¬ 
mance  goals,  which  would  operate  in  at  least 
10  agencies  for  three  years,  beginning  in  fiscal 
year  1994.  The  OMB  Director  would  be  re¬ 
quired  to  report  on  the  results  of  these  pilots 
by  May  1, 1997. 

A  second  set  of  pilot  projects  would  test  the 
concept  of  managerial  flexibility.  These 
projects  would  operate  for  two  years  beginning 
in  fiscal  year  1995;  the  May  1,  1997  OMB  re¬ 
port  would  include  a  discussion  of  them  as 
well. 

Third,  the  bill  would  mandate  pilot  projects 
for  performance  budgeting.  Performance  bud¬ 
gets  would  present  varying  levels  of  perfor¬ 
mance  resulting  from  different  budgeted  lev¬ 
els.  At  least  five  agencies  would  be  required  to 
participate  in  these  pilot  projects,  which  would 
run  for  fiscal  years  1998  and  1999.  The  Direc¬ 
tor  of  OMB  would  report  on  the  results  of  the 
performance  budgeting  pilots  no  later  than 
March  31, 2001. 


Executive  Branch  Proposals 

In  addition  to  the  Chief  Financial  Officers  Act, 
several  efforts  aim  to  change  fundamentally 
the  way  that  federal  services  are  provided; 
some  of  these  are  targeted  to  alter  federal  bud¬ 
geting  as  well.  One  effort,  which  focuses  al¬ 
most  exclusively  on  management  systems,  is 
the  move  toward  total  quality  management  in 
the  federal  government.  The  reform  related 
most  closely  to  the  development  of  perfor¬ 
mance  measures,  however,  is  represented  by 
the  "Reinventing  Government"  movement 
(after  a  book  of  the  same  title).2  The  book  re¬ 


lied  primarily  on  examples  from  state  and  lo¬ 
cal  government  in  calling  for  changes  in  the 
way  government  agencies  are  managed. 

David  Osborne,  one  of  the  authors  of  Re  in¬ 
venting  Government,  subsequently  wrote  a 
chapter  for  the  Progressive  Policy  Institute's 
book  Mandate  for  Change  that  attempted  to 
advise  the  incoming  Clinton  Administration 
about  the  reforms  necessary  to  "reinvent"  the 
federal  government.3  Osborne's  chapter 
called  for  establishing  a  performance-based 
budgeting  system  for  the  federal  government 
that  would  replace  an  emphasis  on  line-item 
control  with  an  emphasis  on  holding  manag¬ 
ers  accountable  for  results;  developing  perfor¬ 
mance  measures  for  federal  programs;  and 
budgeting  on  the  basis  on  performance  tar¬ 
gets,  rewarding  agencies  that  exceed  those 
targets.  In  short,  the  Congress  would  loosen 
its  control  over  inputs  and  would  be  guided  by 
program  outcomes.  This  shift  to  performance- 
based  budgeting  would  be  financed  by  setting 
1  percent  of  the  funds  for  each  program  aside 
to  finance  the  development  of  performance 
measures  (the  strategy  could  represent  bil¬ 
lions  of  dollars  a  year).  The  chapter  also  called 
for  an  overhaul  of  the  civil  service  system, 
sunset  provisions  for  federal  programs,  and 
more  "future-oriented"  budgeting. 

Building  on  the  theme  of  reinventing  gov¬ 
ernment,  President  Clinton  announced  on 
March  3,  1993,  that  Vice  President  Gore  will 
coordinate  a  review  of  all  federal  agencies. 
This  six-month  review  will  identify  current 
federal  activities  that  could  be  discontinued 
and  will  recommend  changes  in  the  way  par¬ 
ticular  federal  programs  are  managed.  Part  of 
the  charge  of  this  review  is  to  develop  a  rec¬ 
ommendation  for  carrying  out  "mission- 
driven,  results  oriented  budgeting."  The  goals 
of  this  effort  closely  mirror  those  included  in 
Osborne’s  chapter  in  Mandate  for  Change.  In 


2.  David  Osborne  and  Ted  Gaebler,  Reinventing  Govern - 
ment  (Reading,  Mass.:  Addison-Wesley,  1992). 

3.  David  Osborne,  "Reinventing  Government:  Creating  an 
Entrepreneurial  Federal  Establishment,”  in  Will  Mar¬ 
shall  and  Martin  Schram,  eds..  Mandate  for  Change 
(Washington,  D.C.:  Progressive  Policy  Institute,  1993). 
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general,  the  focus  is  on  shifting  the  emphasis 
of  budgeting  away  from  line  items  and  spend¬ 
ing  money  toward  a  concentration  on  results 
and  saving  money.  Because  this  process  is  in 
its  early  stages,  it  is  unclear  what  its  outcome 
will  be.  A  final  report,  however,  is  due  in  Sep¬ 
tember  1993. 

In  a  related  effort,  OMB  has  during  the  past 
year  been  staging  "quality  improvement  dem¬ 
onstrations"  at  three  federal  agencies-the  So¬ 
cial  Security  Administration,  the  Internal 
Revenue  Service,  and  the  Department  of  Vet¬ 
erans  Affairs.  The  demonstrations  include 
specific  projects  in  each  agency  and  are  in¬ 
tended,  in  part,  to  develop  more  and  better 
measures  of  program  performance  in  these 
agencies.  OMB  expects  to  obtain  the  results  of 
these  demonstrations  in  one  to  three  years. 


Strategies  for  Improving 
Performance 
Measurement  and 
Performance  Budgeting 

Experience  and  current  practice  provide  in¬ 
sights  that  may  be  useful  in  understanding 
the  prospects  for  performance  measurement 
and  performance  budgeting  in  the  federal  gov¬ 
ernment.  First,  any  effort  to  measure  perfor¬ 
mance  must  confront  the  issue  of  the  appro¬ 
priate  combination  of  executive  and  legisla¬ 
tive  branch  action.  Second,  the  complexity  of 
the  endeavor  suggests  that  a  deliberate  ap¬ 
proach  may  be  most  fruitful.  Third,  it  is  im¬ 
portant  to  understand  how  performance  mea¬ 
sures  might  influence  the  budget  process, 
which  requires  understanding  the  limitations 
of  this  type  of  change  in  immediately  influenc¬ 
ing  policy. 

Legislative  and  Executive 
Activity 

If  performance  measurement  is  to  gain  re¬ 
newed  currency  for  either  management  or 


budgeting,  the  appropriate  combination  of  leg¬ 
islative  and  executive  activity  must  be  ap¬ 
plied.  This  entails  understanding  how  each  is 
limited  in  bringing  about  lasting  changes.  As 
Chapter  2  indicated,  states  have  had  more  dif¬ 
ficulty  than  local  or  foreign  governments  in 
interesting  legislative  bodies  in  performance 
measurement.  The  difficulty  may  derive  in 
part  from  the  nature  of  state  government  or¬ 
ganization.  Like  the  federal  government, 
state  legislative  and  executive  branches  are 
both  more  separated  and  more  fragmented 
than  those  of  local  or  foreign  governments. 
Second-and  related-passing  laws  is  not 
enough;  without  commitment  from  an  execu¬ 
tive  branch,  any  "management"  effort  is 
doomed  to  fail. 

Nonetheless,  passing  legislation  could  yield 
several  potential  benefits.  First,  federal  legis¬ 
lation  can  motivate  the  executive  branch  or 
particular  agencies  to  pursue  changes  that 
they  might  not  pursue  unilaterally.  Second,  if 
increased  managerial  flexibility  is  desirable, 
and  if  changes  in  law  are  necessary  to  provide 
this  flexibility,  the  Congress  can  enact  them. 
Third,  laws  can  suggest  particular  kinds  of  in¬ 
formation  that  the  Congress  would  consider 
desirable  for  decisionmaking.  Fourth,  the 
Congress  can  suggest  areas  that  the  executive 
branch  might  study  or  test,  in  anticipation  of 
more  sweeping  legislation  in  the  future. 

Legislation  has  its  limitations,  however. 
Ultimately,  without  executive  branch  commit¬ 
ment  to  performance  measures  for  agency 
management,  the  practice  will  not  gain  wide¬ 
spread  acceptance.  Further  improving  the  use 
of  performance  measures  for  financial  man¬ 
agement,  financial  reporting,  or  budgeting  im¬ 
plies  commitment  from  executive  branch  bud¬ 
get  offices  (in  agencies  and  OMB)  and  from  the 
committees  in  the  Congress  that  must  change 
their  behavior.  An  important  behavioral 
change  would  involve  a  switch  from  emphasiz¬ 
ing  front-end,  micro-level  inputs  to  emphasiz¬ 
ing  the  big  picture-the  results  obtained  from 
public  programs. 
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A  Deliberate  Approach 
May  Be  Most  Desirable 

Performance  measurement  is  limited  in  its 
ability  to  bring  about  substantial  change. 
Some  of  these  factors  have  nothing  to  do  with 
either  the  limits  of  legislation  or  executive 
branch  commitment,  but  simply  with  the  dif¬ 
ficulty  of  measuring  government  performance 
itself.  Designing  systems  that  appropriately 
link  the  goals  of  programs  with  their  results, 
and  that  link  results  to  budgeting  and  finan¬ 
cial  reporting,  is  an  extraordinarily  complex 
task.  The  case  studies  included  in  this  study 
illustrate  that  the  greatest  obstacle  to  using 
performance  measures  lies  in  first  identifying 
the  measures.  That  is  true  at  the  local  and 
state  level,  but  is  particularly  true  for  the  fed¬ 
eral  government,  where  the  success  of  so  many 
programs  is  influenced  by  other  actors,  includ¬ 
ing  state  and  local  governments,  private  busi¬ 
nesses,  and  individuals.  It  is  hard  to  disagree 
with  the  goal  of  improving  the  measurement 
of  government  performance.  There  is  a  dan¬ 
ger,  however,  in  acting  precipitously.  Exper¬ 
ience  with  the  planning-programming-bud¬ 
geting  system  and  other  reforms  suggests  that 
it  can  be  counterproductive  to  install  far- 
reaching  reforms  without  a  fairly  complete 
understanding  of  their  effects. 

The  pilot  projects  included  in  S.  20  repre¬ 
sent  examples  of  the  type  of  deliberate  ap¬ 
proach  needed  for  deciding  whether  to  carry 
out  a  far-reaching  performance  measurement 
system.  One  approach  to  decision-making  in 
this  instance  would  be  to  use  pilot  projects  to 
collect  information  on  costs  and  potential 
benefits  before  adopting  such  a  system 
throughout  the  government.  These  pilots 
should  include  agencies  that  represent  the  full 
spectrum  of  federal  activities,  since  failure  to 
do  so  may  result  in  invalid  conclusions.  Fur¬ 
thermore,  pilots  should  not  be  limited  to  those 
agencies  that  already  have  systems  in  place, 
or  even  those  with  easily  quantifiable  mea¬ 
sures.  In  fact,  it  is  most  important  to  include 
those  agencies  in  which  it  is  harder  to  identify 
measurable  objectives.  National  defense  and 
research  and  development  are  two  examples. 


Regulatory  or  intergovernmental  activities— 
in  which  it  would  be  difficult  to  sort  out  the  ef¬ 
fects  of  federal  efforts,  as  opposed  to  state,  lo¬ 
cal,  or  private  initiatives-are  another. 

A  final  comment  about  the  difficulty  of  mea¬ 
suring  performance  concerns  the  relationship 
between  performance  measurement  and  pro¬ 
gram  evaluation.  Performance  measures 
should  not  be  viewed  as  ends  in  themselves. 
In  most  cases,  they  can  only  offer  clues  about 
how  well  a  program  is  achieving  its  results.  In 
many  cases,  a  comprehensive  evaluation  of  a 
program  is  necessary  in  order  to  determine 
whether  it  is  operating  successfully.  As  noted 
in  Chapter  4,  determining  the  extent  to  which 
graduates  of  a  job  training  program  have  been 
placed  or  have  retained  jobs  does  not  offer  con¬ 
clusive  evidence  of  the  success  or  failure  of  the 
program.  For  that  reason,  it  is  necessary  to 
understand  the  limitations  of  performance 
measures  and  interpret  them  accordingly. 

Limits  on  Using  Performance 
Measurement  for  Budgeting 

It  is  difficult  to  disagree  with  the  concept  of 
budgeting  based  on  performance.  But  the  ex¬ 
perience  of  other  levels  of  government  sug¬ 
gests  limited  success.  Neither  the  case  studies 
presented  in  this  analysis  nor  those  discussed 
in  the  report  issued  by  the  General  Account¬ 
ing  Office  earlier  this  year  uncovered  exam¬ 
ples  in  which  performance  measures  had  sig¬ 
nificantly  influenced  the  allocation  of  re¬ 
sources.  There  is  little  evidence  that  policy¬ 
makers  use  performance  measures  to  help 
them  make  large  changes  in  the  allocation  of 
resources,  or  that  they  receive  detailed  in¬ 
formation  on  the  relationship  between  re¬ 
sources  and  outcomes.  Where  performance 
measurement  has  taken  hold  in  the  budget 
process-in  some  state  and  local  governments, 
for  example-agency  managers  tend  to  use 
measurements  to  manage  their  budgets. 
Thus,  performance  measures  are  used  much 
more  extensively  in  carrying  out  the  budget 
than  in  preparing  it. 
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The  fact  that  performance-based  budgeting 
has  not  gained  widespread  acceptance  is  not 
reason  enough  to  discount  its  potential.  Even 
if  both  legislative  and  executive  branches 
were  committed  to  it,  and  a  genuine  effort  was 
undertaken  to  transform  government  budget¬ 
ing,  there  would  still  be  obstacles.  It  is  not 
clear  how  performance  measures  should  be 
used  to  allocate  resources.  For  example,  one 
cannot  simply  reward  those  agencies  whose 
measures  indicate  "good”  performance  and 
take  resources  away  from  those  whose  mea¬ 
sures  indicate  "bad”  performance.  Reality  is 
certainly  much  more  complicated  than  that.  A 
thorough  understanding  of  all  of  the  factors 
(including  the  level  of  funding)  that  contribute 
to  performance  is  necessary  in  order  to  under¬ 
stand  how  performance  measures  can  be  used 
to  allocate  resources. 

Furthermore,  performance  measures  sel¬ 
dom  make  the  task  of  choosing  between  dif¬ 
ferent  uses  of  public  resources  easier.  Ulti¬ 
mately,  if  every  program  had  performance 
measures,  policymakers  could  understand  the 
trade-offs  inherent  in  spending  money  on  two 
competing  programs.  For  example,  if  the 
choice  was  between  a  job  training  and  an  air 
pollution  program,  we  might  know  that  add¬ 
ing  $100  million  more  to  the  Environmental 
Protection  Agency  budget  would  make  the  air 
cleaner  by  X  amount  but  would  cost  Y  amount 
of  lost  wages  from  workers  who  had  not  been 
trained.  If  all  of  these  data  were  available 
(and  believable),  that  would  make  decisions 
more  informed.  But  it  still  would  not  neces¬ 
sarily  make  the  choices  less  difficult. 

Budgeting  based  on  performance  flies  in  the 
face  of  existing  budgeting  practice.  A  system 
that  affords  less  control  over  individual  line 
items  in  order  to  hold  agencies  solely  account¬ 
able  for  results  would  be  a  fundamental 
change  from  the  current  system.  Such  a  sys¬ 
tem  could  not  resolve  the  issue  of  how  much 
money  goes  to  the  defense  budget  and  how 
much  goes  to  domestic  spending  solely  by  us¬ 
ing  performance  measures.  Budgeting,  in  a 
democratic  society,  is  inherently  political.  No 
set  of  budget  techniques  can  substitute  for  po¬ 
litical  decisions  about  "who  wins"  and  "who 


loses."  In  fact,  the  failures  of  past  efforts,  such 
as  PPBS,  are  largely  the  result  of  an  inability 
to  account  for  this  shortcoming. 

The  limited  potential  of  performance  mea¬ 
sures  to  influence  budget  outcomes  directly, 
however,  does  not  mean  that  they  have  no 
place  in  the  budget  process.  There  are  at  least 
two  other  ways  that  performance  measures 
can  be  tied  to  the  management  of  resources. 
Performance  measures  can  be  used  to  assist 
agencies  in  the  management  of  a  relatively 
fixed  level  of  resources.  For  example,  an  agen¬ 
cy's  total  funding  level  may  be  fairly  stable, 
but  it  may  use  a  performance  measurement 
system  to  allocate  funds  among  geographic  or 
functional  subunits. 

Performance  measures  can  also  be  used  to 
present  information  on  the  relationship  be¬ 
tween  inputs  and  outcomes.  That  would  de¬ 
fine  for  policymakers  the  relationship  between 
given  levels  of  inputs  and  the  results  expected 
from  them.  For  example,  the  Internal  Rev¬ 
enue  Service  might  present  information  on 
how  much  faster  it  could  process  income  tax 
refunds  for  each  of  several  different  levels  of 
funding.  But  it  is  one  thing  to  present  such  in¬ 
formation  to  decisionmakers;  it  is  quite  an¬ 
other  for  them  to  use  it. 

If  performance  measures  are  to  be  used  to 
influence  the  allocation  of  resources,  the 
change  is  not  likely  to  happen  suddenly. 
Rather,  it  may  be  the  result  of  a  change  in  cul¬ 
ture  that  starts  with  the  development  of  bet¬ 
ter,  valid  performance  information  at  the 
agency  level  and  with  the  reporting  of  that  in¬ 
formation  for  nonbudgetary  purposes.  Once 
this  information  is  in  the  public  domain,  it 
might  be  more  accepted  and  eventually  used 
by  decisionmakers  as  they  allocate  resources. 
If  this  change  has  the  effect  of  forcing  some 
spending  to  face  greater  scrutiny,  it  certainly 
is  desirable. 

As  more  attempts  are  made  to  tie  perfor¬ 
mance  measures  to  the  budget,  using  the  right 
measurements  and  collecting  accurate  infor¬ 
mation  become  more  important.  But  the  high¬ 
er  the  stakes  are,  the  greater  the  incentives 
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are  for  people  to  identify  self-serving  mea¬ 
sures  and  report  misleading  data.  Without  a 
process  of  ensuring  that  the  right  measures 
are  chosen  and  reported  accurately,  perfor¬ 
mance  measurement  will  never  deliver  on  its 
promise.  When  performance  measures  be¬ 
come  only  a  more  sophisticated  means  of  agen¬ 
cy  justification,  they  cease  to  be  a  useful  poli¬ 
cymaking  tool.  The  obvious  implication  is 
that  verifying  the  accuracy  of  reported  data  is 
an  essential  part  of  the  measurement  process. 

The  limited  potential,  in  the  short  run,  for 
performance  measures  to  influence  the  alloca¬ 
tion  of  resources  should  not  discourage  the 
Congress  and  the  executive  branch  from  con¬ 
tinuing  to  concentrate  more  resolutely  on  the 
results  of  public  programs.  In  fact,  the  great¬ 
est  reward  to  be  gained  from  the  use  of  perfor¬ 
mance  measures  may  have  less  to  do  with 
governmentwide  budgeting  than  with  the 
task  of  using  existing  resour  ces  to  improve 
performance.  For  example,  performance  mea¬ 
sures  can  be  useful  as  motivational  tools;  that 
is,  they  can  encourage  people  to  achieve  per¬ 
formance  targets.  Ultimately,  repeated  use 
and  exposure  will  result  in  the  development  of 
a  culture  of  performance.  Performance  targets 
may  not  be  precisely  correct  at  first,  and  the 
measurements  may  not  be  either.  But  encour¬ 
aging  federal  managers  and  employees  to 
think  in  terms  of  outcomes  rather  than  inputs 
or  outputs  will  produce  desirable  results.  Be¬ 
cause  the  measurements  will  not  be  right  at 
first,  one  must  be  realistic  about  how  much 
they  can  be  used  to  influence  budgeting  in  the 
near  term.  In  short,  improving  agency  perfor¬ 
mance  measurement  (because  agencies  should 
increase  their  ability  to  get  the  measures  right 
over  time)  should  precede  using  those  mea¬ 
sures  for  budgeting. 


Conclusion 

Before  taking  further  legislative  action,  it  is 
important  for  the  Congress  to  consider  exactly 
how  it  expects  performance  measures  to 
change  the  process  of  developing  and  carrying 
out  policy.  If  mandating  performance  mea¬ 
sures  is  expected  by  itself  to  reform  federal 
budgeting  and  management,  disappointment 
will  almost  certainly  follow.  Executive  branch 
commitment  is  probably  a  more  important  fac¬ 
tor  than  legislation,  although  legislation 
aimed  at  reducing  impediments  to  managers 
might  be  helpful  as  well.  Further,  several  con¬ 
tinuing  efforts  touch  on  performance  measure¬ 
ment:  total  quality  management,  carrying 
out  the  CFO  act,  deliberations  of  the  Federal 
Accounting  Standards  Advisory  Board,  legis¬ 
lative  efforts  such  as  S.  20,  and  the  Vice  Presi¬ 
dent's  performance  review.  These  efforts 
should  be  viewed  as  related  to  each  other  and 
should  be  coordinated.  In  particular,  the  effort 
to  identify  agency  missions  and  goals  should 
precede  the  effort  to  arrive  at  measures  of  per¬ 
formance,  and  the  development  of  perfor¬ 
mance  measures  should  precede  requiring 
their  use  for  budgeting. 

In  the  final  analysis,  the  budget  process  is 
unlikely  to  be  changed  substantially  until  and 
unless  decisionmakers  demand  and  use  in¬ 
formation  on  program  performance  when 
making  decisions  about  allocating  resources. 
Having  this  information-if  good  measures  of 
program  success  can  be  developed-is  a  neces¬ 
sary,  but  not  sufficient,  prerequisite  to  chang¬ 
ing  the  policy  process. 


